Generalizable Person Re-identification Without Demographics

Abstract: Generalizable Person Re-Identification (DG ReID) aims to learn ready-to-use cross-domain representations for direct cross-data evaluation. It typically fully exploit demographics information, e.g. the domain information and camera IDs to learn features that are domain-invariant. However, the protected demographic features are not often accessible due to privacy and regulation issues. Under this more realistic setting, distributionally robust optimization (DRO) provides a promising way for learning robust models that are able to perform well on a collection of possible data distributions (the ``uncertainty set”) without demographics. However, the convex condition of KL DRO may not hold for overparameterized neural networks, such that applying KL DRO often fails to generalize under distribution shifts in real scenarios. Instead, by applying the change-of-measure technique and the analytical solution of KL DRO, we propose a simple yet efficient approach, Unit DRO. Unit DRO minimizes the loss over a reweighted dataset where important samples (i.e. samples on which models perform poorly) will be upweighted and others will be downweighted. Empirical results show that Unit DRO achieves superior performance on large-scale DG ReID and cross-domain ReID benchmarks compared to standard baselines.

Reviews and author responses:

 Thanks for your constructive comments! We conduct experiments in general DG benchmarks. Comprehensive experiments show that, for general DG tasks, UnitDRO consistently outperforms all the baselines by a considerable margin even without demographics.


|   Domain  |            |            |    PACS    |            |      |            |            |    VLCS    |            |      |
|:---------:|:----------:|:----------:|:----------:|:----------:|:----:|:----------:|:----------:|:----------:|:----------:|:----:|
|           |      A     |      C     |      P     |      S     |  Avg |      C     |      L     |      S     |      V     |  Avg |
|    IRM    | 85.7 ± 1.0 | 79.3 ± 1.1 | 97.6 ± 0.4 | 75.9 ± 1.0 | 84.6 | 97.6 ± 0.5 | 64.7 ± 1.1 | 69.7 ± 0.5 | 76.6 ± 0.7 | 77.2 |
| Group DRO | 88.2 ± 0.7 | 82.4 ± 0.8 | 97.7 ± 0.2 | 80.6 ± 0.9 | 87.2 | 97.8 ± 0.0 | 66.4 ± 0.5 | 68.7 ± 1.2 | 76.8 ± 1.0 | 77.4 |
|   MIXUP   | 87.4 ± 1.0 | 80.7 ± 1.0 | 97.9 ± 0.2 | 79.7 ± 1.0 | 86.4 | 98.3 ± 0.3 | 66.7 ± 0.5 | 73.3 ± 1.1 | 76.3 ± 0.8 | 78.7 |
|    DANN   | 86.4 ± 1.4 | 80.6 ± 1.0 | 97.7 ± 0.2 | 77.1 ± 1.3 | 85.5 | 95.3 ± 1.8 | 61.3 ± 1.8 | 74.3 ± 1.0 | 79.7 ± 0.9 | 77.7 |
|  Unit DRO | 88.3 ± 0.1 | 84.8 ± 0.1 | 96.4 ± 0.1 | 82.1 ± 0.1 | 87.9 | 98.1 ± 0.1 | 68.0 ± 0.0 | 71.3 ± 0.1 | 78.8 ± 0.0 | 79.1 |

Datasets: PACS [6] and VLCS [7]. 

Baselines:   We compare our model with IRM [2], Group DRO [3],  DANN [4], and an augmentation method Mixup [5].

We repeat the entire experiment three times using different seeds to reduce the randomness. Finally, we report the mean over these repetitions as well as their estimated standard error. We use test-domain validation, one of the three selection methods in [1]. This strategy is an oracle selection one since we choose the model maximizing the accuracy on a validation set that follows the distribution of the test domain.

[1] In Search of Lost Domain Generalization, ICLR, 2021

[2] Invariant Risk Minimization, Arxiv, 2019

[3] Distributionally Robust Neural Networks, ICLR, 2020

[4] Domain-adversarial training of neural networks, JMLR, 2016

[5]  Improve unsupervised domain adaptation with mixup training, Arxiv, 2020

[6] Deeper, broader, and artier domain generalization, ICCV, 2017

[7] Un biased  look  at dataset bias, CVPR, 2011

 Thanks for your constructive comments! We conduct experiments in general DG benchmarks. Comprehensive experiments show that, for general DG tasks, UnitDRO consistently outperforms all the baselines by a considerable margin even without demographics.


|   Domain  |            |            |    PACS    |            |      |            |            |    VLCS    |            |      |
|:---------:|:----------:|:----------:|:----------:|:----------:|:----:|:----------:|:----------:|:----------:|:----------:|:----:|
|           |      A     |      C     |      P     |      S     |  Avg |      C     |      L     |      S     |      V     |  Avg |
|    IRM    | 85.7 ± 1.0 | 79.3 ± 1.1 | 97.6 ± 0.4 | 75.9 ± 1.0 | 84.6 | 97.6 ± 0.5 | 64.7 ± 1.1 | 69.7 ± 0.5 | 76.6 ± 0.7 | 77.2 |
| Group DRO | 88.2 ± 0.7 | 82.4 ± 0.8 | 97.7 ± 0.2 | 80.6 ± 0.9 | 87.2 | 97.8 ± 0.0 | 66.4 ± 0.5 | 68.7 ± 1.2 | 76.8 ± 1.0 | 77.4 |
|   MIXUP   | 87.4 ± 1.0 | 80.7 ± 1.0 | 97.9 ± 0.2 | 79.7 ± 1.0 | 86.4 | 98.3 ± 0.3 | 66.7 ± 0.5 | 73.3 ± 1.1 | 76.3 ± 0.8 | 78.7 |
|    DANN   | 86.4 ± 1.4 | 80.6 ± 1.0 | 97.7 ± 0.2 | 77.1 ± 1.3 | 85.5 | 95.3 ± 1.8 | 61.3 ± 1.8 | 74.3 ± 1.0 | 79.7 ± 0.9 | 77.7 |
|  Unit DRO | 88.3 ± 0.1 | 84.8 ± 0.1 | 96.4 ± 0.1 | 82.1 ± 0.1 | 87.9 | 98.1 ± 0.1 | 68.0 ± 0.0 | 71.3 ± 0.1 | 78.8 ± 0.0 | 79.1 |

Datasets: PACS [6] and VLCS [7]. 

Baselines:   We compare our model with IRM [2], Group DRO [3],  DANN [4], and an augmentation method Mixup [5].

We repeat the entire experiment three times using different seeds to reduce the randomness. Finally, we report the mean over these repetitions as well as their estimated standard error. We use test-domain validation, one of the three selection methods in [1]. This strategy is an oracle selection one since we choose the model maximizing the accuracy on a validation set that follows the distribution of the test domain.

[1] In Search of Lost Domain Generalization, ICLR, 2021

[2] Invariant Risk Minimization, Arxiv, 2019

[3] Distributionally Robust Neural Networks, ICLR, 2020

[4] Domain-adversarial training of neural networks, JMLR, 2016

[5]  Improve unsupervised domain adaptation with mixup training, Arxiv, 2020

[6] Deeper, broader, and artier domain generalization, ICCV, 2017

[7] Un biased  look  at dataset bias, CVPR, 2011

 
(i) Thank you for pointing out the question! 1) The specialized domain information of Market1501 and CUHK is described by the author in the article, such as the campus of Tsinghua University and the Chinese University of Hong Kong. In practice, different from these public datasets for research purposes, ReID domain/cameras information acquisition is often completed automatically, based on some inherent physical properties of cameras (such as network cameras MAC address). With the increasing development of the Internet of Things, the utilization of cameras’ physical properties will definitely increase the leakage risk of location information, as each device will be online in the future.  2) Even if the location information is eliminated, the relationships between different people are still exposed by the annotated domain information. If two-person IDs are marked with the same camera, their social relationships can be simply measured by counting their occurrence frequency in the same camera. So the private social relationships among persons may be leaked, which may be more sensitive than location information. In summary, removing these domain/camera information in data collection is definitely a safer and efficient usage mode of the person images. We argue that the proposed DGWD setting is undoubtedly reasonable and valuable for the privacy-sensitive ReID task.

(ii) MixStyle [4] is a powerful augmentation method, which is orthogonal to all studies and cannot eliminate the value of DGWD-ReID. We believe separating DGWD-ReID from DG-ReID is very meaningful. As we said before, more and more DG methods use demographics to learn invariant features. Although some augmentation or normalization methods can be used when demographics are forbidden. These mentioned studies just do not **rely on** demographics, however, do also not **focus on** DG without demographics. [4] can also improve their performance by using domain labels. In contrast, DGWD-ReID adds a strict restriction on demographics and has ambitious targets that **Can we learn more invariant features even without demographics?** and **Can we partition domains better?**.

(iii) Previous we get used to considering these auto-generated domain labels, camera labels are optimal for learning invariant representations. However, label errors are ubiquitous in most datasets [1]. Besides, previous studies show that how best to partition domains [3] that can benefit generalization capability most is still unclear. In the fairness community, existing work has found that designing rational methodology can found domains that are maximally informative for downstream invariant learners.  These domain IDs, camera IDs make sense for humans, however, the similarities between different domains/camera views vary greatly. Can we find more powerful domain partitions for ReID tasks is still an open problem.  

[1] Pervasive label errors in test sets destabilize machine learning benchmarks, Arxiv, 2021

[2] Environment inference for invariant learning. ICML, 2021

[3] Robustness to Spurious Correlations via Human Annotations. ICML 2020.

[4] Domain Generalization with MixStyle, ICLR 2021

 Thanks for your detailed response.I agree that the concept of DGWD-ReID is still doubtful. 

Firstly, the authors claim that camera ID and frame stamps contain information that can be used to track a pedestrian easily, but this information can be simply desensitized. For example, it is almost impossible to trace back the location captured in images in Market-1501 if it was not mentioned in the original paper or being desensitized properly. Moreover, some of the existing works [1] that normalize style do not utilize any side information mentioned by the authors. Therefore, the privacy concerns mentioned by the authors might not hold for DGWD-ReID setting.

Secondly, leveraging camera ID is proven to be an efficient method for domain generalization in previous works [2][3]. The authors claim that it "may be sub-optimal for neural networks learning invariant and generalizable Re-ID features", please proof that. 
Finally, the proposed work is more likely to be a general DG framework rather than being designed for Re-ID. It is better to should the improvements on other DG tasks.

In conclusion, the DGWD-ReID setting seems like should not be considered independently from DG-ReID. It is more like a special case of DG-ReID. And the author does not convince me. The claimed novelty of the problem setting is a bit oversold.

[1] Domain Generalization with MixStyle, ICLR 2021

[2] Rethinking the Distribution Gap of Person Re-identification with Camera-based Batch Normalization, ECCV, 2020

[3] ICE: Inter-instance Contrastive Encoding for Unsupervised Person Re-identification, ICCV, 2021


This paper introduces a new domain generalizable person re-identification method and a new setting. The authors propose an interesting setting, solving domain generalizable Re-ID tasks without using demographic information (DGWD-ReID). This new setting is based on the concern of privacy risk for safe-critical applications. The paper introduces Unit DRO to construct the uncertainty set for possible distributions of unseen domains and reweight samples to learn robust domain-invariant features under the new setting.

 Advantages:
The problem setting is interesting, and the proposed method is novel. It does not require the demographic knowledge, which is used in most generalizable person Re-ID methods, nor predefined distribution shifts assumed in Fairness problems. The proposed Unit DRO method overcomes the problem of optimizing on implausible worst-case distributions under DGWD-ReID setting. The authors demonstrate the effectiveness of the method and shows the impact of hyperparameters with abundant experiments. The performance of the proposed method surpasses SOTA methods under various settings.

Drawbacks:
1.	There is a formulation error in METHODOLOGIES.
2.	There are a few typo problems.
3.	The method is highly hyperparameter sensitive.
4.	The method is vulnerable to against relatively large camera perspective changes


 Questions:
1.	The meaning of this DGWD-ReID setting is doubtful. Most public datasets not only contain person ID but cam ID as well. And the numerical domain labels are hard to expose geographical locations and environment information.
2.	The authors claim that "KL DRO is not held for over-parameterized neural networks", please provide a demonstrate on that.
3.	As the problem mentioned in Drawbacks.3, is there an easy way to find the optimized hyperparameters for the proposed framework? Or is there a rule for these hyperparameters?

 1, Overselling the concept of DGWD-ReID

 (I)  Domain labels are all demographics, so the mentioned [4, 5, 6, 7] cannot apply to DGWD-ReID directly. We believe separating DGWD-ReID from DG-ReID is very meaningful. The main reason is depicted in point 1 of reviewer umsc. The most important reasons are listed as follows: (1) The domain labels, camera ids are all automatically generated and may be sub-optimal for neural networks learning invariant and generalizable ReID features. It is promising to inference more informative demographics by the neural network itself, rather than using these auto-generated labels. (ii) Without domain labels, researchers will be forced to exploit invariant features from the training data itself, rather than the side information (manually divided domains, cameras, and frames). Thus, the proposed DGWD-ReID setting is more common and challenging. Although mentioned normalization and augmentation methods [3, 9] and many other studies can be applied to DGWD-ReID, it is still valuable to propose DGWD-ReID. **These mentioned studies do not rely on demographics, however, do also not focus on DG without demographics.** [9] can also improve their performance by using domain labels. In contrast, DGWD-ReID adds a strict restriction on demographics and has ambitious targets that **Can we learn more invariant features even without demographics?**, **Can we partition domains better?** 

(ii) We believe that normalization and augmentation are of equal importance for DG ReID. Here we use tricks just to show that these methods are orthogonal to other methods. These normalizations and augmentation methods can also be used in Open-set DA [1] and Partial DA [2], closed-set DA, and many other settings, however, we think it is depressed that ignore the meanings of these settings. Besides, It is unclear why [4] is referenced here, there are many other studies that try to learn domain invariant features. As far as we know, learning more invariant representations is important for both DA and DG settings.

(iii) This question is also mentioned in point 1 of reviewer umsc.  ReID is research about a person and is mostly used in security systems, and naturally with higher requirements for privacy [10]. In current DG ReID systems, each domain corresponds to a certain location (CUHK02, 03<->CUHK). Camera id corresponds to a certain view and contains more detailed location information. Frames contain temporal information. Using this information, the trace of one pedestrian is easy to monitor and predict. Hence, it is of vital importance for trying to avoid including this real information in the datasets.

(2,3). Due to the project continuity and previous background in our group, we chose the ReID task to verify the proposed methods. We will expand the potentials of the proposed methods with more OOD tasks in future work. Thank you for the constructive ideas. In our experiments, enforcing Unit DRO on classification loss and the triplet loss is almost the same as only on classification loss. Finally, the triplet loss is not unique for ReID tasks, there are many ReID studies that do not use the triplet loss [13] [14] and triplet loss is commonly used in DA [11] and DG [12]. 

4, We agree that IN can boost the performance in DG ReID by a large margin and most current DG-ReID methods incorporate these modules, so do us.  We used the normalization method in [5], but the baseline is not exactly the method in [5] ([5] uses the domain label), so in the more challenging setting, we surpassed the method in [5]. The performance is sufficient to show the effectiveness of the method. At the same time, using the SOTA method as a baseline is not a problem, most innovations stand on the shoulders of gaints! 

[1] Open set domain adaptation, ICCV 2017.

[2] Partial transfer learning with selective adversarial networks, CVPR 2018.

[3] Style Normalization and Restitution for Generalizable Person Re-Identification, CVPR 2020.

[4] Generalizable Person Re-identification by Domain-Invariant Mapping Network, CVPR 2019.

[5] Meta Batch-Instance Normalization for Generalizable Person Re-Identification, CVPR 2021.

[6] Learning to Generalize Unseen Domains via Memory-based Multi-Source Meta-Learning for Person Re-Identification, CVPR 2021.

[7] Generalizable Person Re-identification with Relevance-aware Mixture of Experts, CVPR 2021.

[9] Domain Generalization with MixStyle, ICLR 2021.

[10] Performance Optimization for Federated Person Re-identification via Benchmark Analysis, ACMMM, 2020

[11] Rethinking Triplet Loss for Domain Adaptation, in IEEE Transactions on Circuits and Systems for Video Technology, Jan. 2021

[12] Learning from extrinsic and intrinsic supervisions for domain generalization, ECCV, 2020

[13] Towards Rich Feature Discovery with Class Activation Maps Augmentation for Person Re-Identification, CVPR, 2019

[14] Joint Discriminative and Generative Learning for Person Re-identification, CVPR, 2019


 In view of the current research trend, more and more DG methods use demographics to learn invariant features. Our setting is more challenging and more general. Besides, the current evaluation standards of DG-ReID are not uniform, and many settings and evaluation methods are being explored. We have added experiments on all existing protocols (detailed in point 2 of the reviewer #eLS6 ), we hope that unifying these evaluation standards is beneficial to the development of DG.

|    Method   |  Source |  Domain labels  | Camera labels |
|:-----------:|:-------:|:---------------:|:-------------:|
| MetaBIN [1] | CVPR 21 |        √        |       √       |
|   M3L [2]   | CVPR 21 |        √        |               |
| DMG-Net [3] | CVPR 21 |        √        |       √       |
|  RaMoE [4]  | CVPR 21 |        √        |               |
|   CBN [5]   | ECCV 20 |                 |       √       |
|     [6]     | ECCV 20 |        √        |       √       |
|  QAConv [7] | ECCV 20 | Strong baseline |               |
|    SNR[8]   | CVPR 20 | Strong baseline |               |
|   DIMN [9]  | CVPR 19 |        √        |       √       |

[1] Meta Batch-Instance Normalization for Generalizable Person Re-Identification, CVPR, 2021

[2] Learning to Generalize Unseen Domains via Memory-based Multi-Source Meta-Learning for Person Re-Identification, CVPR, 2021

[3] Person30K: A Dual-Meta Generalization Network for Person Re-Identification, CVPR, 2021

[4] Generalizable Person Re-Identification With Relevance-Aware Mixture of Experts, CVPR, 2021

[5] Rethinking the Distribution Gap of Person Re-identification withCamera-based Batch Normalization, ECCV, 2020

[6] Generalizing Person Re-Identification by Camera-Aware Invariance Learning and Cross-Domain Mixup, ECCV, 2020

[7] Interpretable and Generalizable Person Re-identification with Query-adaptive Convolution and Temporal Lifting, ECCV, 2020

[8] Style Normalization and Restitution for Generalizable Person Re-Identification, CVPR, 2021

[9] Generalizable Person Re-identification by Domain-Invariant Mapping Network, CVPR, 2019

 Thanks for your response. I have read your response carefully and read your paper twice, but some of the authors’ response has still not solved my concerns and there are other problems. As a result, I will keep on my original score of “3”. I hope to discuss more with other reviewers. The detailed reasons are as follows.

1. Overselling the concept of DGWD-ReID
* (1) The authors take “Open-set DA and Partial DA” as an example to argue that the concept of DGWD-ReID is not oversold. However, such analogy is not accurate. Because Open-set DA [1] and Partial DA [2] differ from Closed-set DA at “the source and target domains do not share the same label space”, and this difference leads to the difference in their designed methods. For another word, Closed-set DA methods cannot be directly applied to Open-set DA or Partial DA. But in DG ReID, domain labels, camera IDs, and video timestamps are not necessary. Taking the existing DG ReID works as an example, [3, 9] does not utilize all the above demographics, [4, 5, 6, 7] only utilize the domain labels to split the training data. Specifically, these DG ReID methods [5, 6, 7] utilize the meta-learning which needs the domain labels to split training domains. However, the problem of DG ReID does not restrict the use of demographics. Besides, refer to the general DG task [8], using domain labels or not using domain labels are both included into DG. As a result, DGWD-ReID should not be independently from DG-ReID and DGWD-ReID is only a kind of special case of DG-ReID. 
 * (2) Why do the authors think the normalization and augmentation [3, 9] are only tricks to enlarge data diversity? Enlarging data diversity and learning invariant representations are two different models to tackle the problem of DG ReID. It is unfair to regard the normalization and augmentation methods as tricks and overemphasize the importance of learning invariant representations. Besides, [4] has already proposed to learning invariant representations in DG ReID.

* (3) Whether the domain labels, camera IDs, and video timestamps will really lead to privacy risks is still a problem that needs to be discussed about. Because all the above demographics can be desensitized like the person IDs.

2.	Some of the authors’ response has solved my concerns. However, the proposed method seems like a method not aiming to solve DG ReID but aiming to solve general DG problems like classification [8] in PACS, Office-Home. Compared with general DG problems, the triplet loss is unique to DG ReID (because the classification loss is used in both general DG and DG ReID), and my concern is why only enforcing DRO on the classification loss instead of the triplet loss. Also, no experiments on general DG tasks like PACS and Office-Home have been conducted.

3.	The magnitude of the loss is not the reason that the authors only enforce Unit DRO on classification loss instead of the triplet loss.

4.	There is another concern that I want the authors to response. I notice that the authors integrate the mixture of Batch Normalization (BN) and Instance Normalization (IN) with learnable parameters [5] in the baseline as the authors said in Page-3 of the original paper. The experiments are only conducted on this baseline, which is not enough to show the superiority of the proposed method. As we all know, a strong baseline in ReID can bring about huge performance gains especially this mixture of normalization (i.e., BN and IN). Now that the authors’ response “normalization and augmentation are only tricks”, why do the authors choose to integrate this kind of tricks in the baseline? Frankly speaking, IN can boost the performance in DG ReID by a large margin. As a result, it is hard to say the performance improvement mainly comes from the proposed method or the strong baseline.


[1] Open set domain adaptation, ICCV 2017.

[2] Partial transfer learning with selective adversarial networks, CVPR 2018.

[3] Style Normalization and Restitution for Generalizable Person Re-Identification, CVPR 2020.

[4] Generalizable Person Re-identification by Domain-Invariant Mapping Network, CVPR 2019.

[5] Meta Batch-Instance Normalization for Generalizable Person Re-Identification, CVPR 2021.

[6] Learning to Generalize Unseen Domains via Memory-based Multi-Source Meta-Learning for Person Re-Identification, CVPR 2021.

[7] Generalizable Person Re-identification with Relevance-aware Mixture of Experts, CVPR 2021.

[8] Domain Generalization in Vision: A Survey, 2021.

[9] Domain Generalization with MixStyle, ICLR 2021.







 | Method |   Avg  |        | Target:PRID |        | Target:GRID |        | Target:VIPeR |        | Target:iLIDs |        |
|:------:|:------:|:------:|:-----------:|:------:|:-----------:|:------:|:------------:|:------:|:------------:|:------:|
|        |   mAP  | Rank-1 |     mAP     | Rank-1 |     mAP     | Rank-1 |      mAP     | Rank-1 |      mAP     | Rank-1 |
|   SNR  |  64.55 | 55.375 |      60     |   49   |     41.3    |  30.4  |      65      |  55.1  |     91.9     |   87   |
|  RaMoE |  71.3  | 63.025 |     66.8    |  56.9  |     53.9    |  43.4  |     72.2     |  63.4  |     92.3     |  88.4  |
|  Ours  | **76.125** | **68.025** |     **79.4**    |  **71.3**  |     **59.8**    |  **50.2**  |     **77.1**     |  **68.9**  |     88.2     |  81.7  |

We can see Unit DRO outperforms existing studies by a large margin.

All of these additional experimental results are attached to the appendix.

 1. You have raised an important point; however, we totally disagree that the concept of generalizable person ReID without demographics is oversold. The meanings of DGWD-ReID setting have been acknowledged by the comments of the other reviewers, e.g., "3Xjf, eLS6". The benefits brought by DGWD-ReID are detailed in response 1 for reviewer umsc, please refer to this response. Here we counter this argument on three points. (1)**We think the logic in this comment is flawed.** For example, Open set DA [8] and Partial DA [3] are both variants of domain adaptation (DA), which constrains the label sets of source and target domains. They can also be included in DA setting. However, you cannot ignore the significance of these settings. **It is always valuable and worth advocating to depict and formalize an existing problem with more accurate boundary lines than previous work.** Although we only take one small step forward, we believe it is beneficial for the ReID community. (2) The mentioned studies [1] and [2] are all normalization/augmentation methods. These methods are plug-and-play modules, which are orthogonal to the Unit DRO models and can be incorporated with our methods to further boost the performance. However, it does not mean that the DGWD-ReID is meaningless. DGWD-ReID enforces the demographics cannot be used, researchers then need to design more efficient methods to guide the training process or learn to partition domains. Namely, **the methods for DGWD-ReID will have to learn invariant representations from training data itself, rather than adopt normalization and augmentation tricks to enlarge data diversity.** As presented in Section 4.3, the proposed Unit DRO different from this normalization method can learn weights that reveal the difficulties of domains (refer to Section 4.3 *Domain divergence measure using $\mathcal{A}$-distance and MMD-distance.*). (3) We emphasize that ReID applications have many higher requirements for privacy.

2. (1) The un-cited DRO-related studies. In your mentioned survey [4] on Out-Of-Distribution (OOD) generalization, the authors divide DRO into different groups by the uncertainty set. We also follow this taxonomy to introduce related work on DRO. (Please refer to extended related work in Section. A of the appendix). We believe the most relevant studies on DRO with ours have been cited in our manuscript. We would appreciate it if you can give some concrete recommendations on related work. (2) Why DRO can help tackle the DG ReID problem? It is detailed both in your mentioned survey [4] and the introduction of this manuscript. We attach the most intuitive idea here ``Since DRO methods  directly optimize for the worst-case risk  within the uncertainty set, as long as the  potential testing distribution, the OOD  generalization ability is guaranteed.'' [4] (3) We agree that DRO can be applied to DG ReID directly, however, with poor performance. Even the well-designed Group DRO attains poor performance in ReID setting. Moreover, it needs additional domain labels to construct regularization terms. Compared to traditional DRO methods, Unit DRO reformulates KL DRO to an important sampling problem and scales well to neural networks. (4) Finally, we disagree that "the proposed method is not specifically designed for DG ReID" is a drawback. Not dependent on any particular structure or task-specific information is exactly Unit DRO's strength (the general framework get positive comment by the review eLS6 ). We focus on ReID tasks just because of the previous background in our group. The researchers including us may expand the potentials of the proposed methods with more OOD tasks in future work.

3. In our experiments, we find that triplet loss is usually smaller than cross-entropy loss by almost two orders of magnitude and thus has little impact on the results.

4. Thanks for the valuable suggestion! We follow the most commonly used DG ReID settings, so we do not include all the protocols in the paper. Here we have included new experiments about all the mentioned settings (Please refer to point 2 of reviewer eLS6). Besides, we have also added a new setting mentioned in [1]. Unit DRO shows promising performance in all these settings. 

5. Thanks for your suggestion again. We will cite these two studies in the final version.

[1] Style Normalization and Restitution for Generalizable Person Re-Identification, CVPR 2020

[2] Domain Generalization with MixStyle, ICLR 2021

[3] Partial transfer learning with selective adversarial networks, CVPR 2018

[4] Towards Out-Of-Distribution Generalization: A Survey, 2021

[5] Distributionally Robust Neural Networks for Group Shifts, ICLR 2020

[6] Generalizable person re-identification with a relevance-aware mixture of experts, CVPR 2021

[7] Learning to generalize unseen domains via memory-based multi-source meta-learning for person re-identiﬁcation, CVPR 2021

[8] Open set domain adaptation. ICCV 2017


 1. We agree with your assessment and it is a very valuable query. Our framework is indeed general and can be applied to more generalization tasks. Here we choose ReID tasks for three reasons (1) ReID applications are very private and usually under privacy issues. ReID datasets are about human, the domains, camera ids correspond to locations, frames indicates time. Once this ground truth information is exposed, one can easily monitor and forecast the activity event of specific identities. For privacy, eliminating demographics in DG-Reid is of vital importance. (2) Generalization ReID is generally considered to be challenging than traditional DG. For DG ReID tasks, the classes in each domain are totally different and styles of images within a domain are of great variance (different cameras). As in the paper, the representative DG method Group DRO does not perform well. (3) Due to the project continuity and previous background in our group, we chose the ReID task to verify the proposed methods. We will expand the potentials of the proposed methods with more OOD tasks in future work. Thank you for the constructive ideas again. 

4. We follow the most commonly used DG ReID settings, so we do not include all the protocols in the paper. We have added new experiments considering the mentioned evaluation protocols. Besides, we also evaluate Unit DRO in the setting mentioned in [1]. All of these results verify the efficiency of Unit DRO. We detail the different evaluation protocols settings as follows. (i) One-to-multiple setting mentioned in [1]. (ii) Multiple-to-one setting mentioned in [2]. (iii) Multiple-to-one setting mentioned in [3].  (iv) Multiple-to-multiple settings are mentioned in [1], which is similar to ours while using different source domains. The detailed difference of these protocols are included in the following table (M:market1501, C2: Cuhk02, C3: Cuhk03, D: DukeMTMC, MT: MSMT17, CS: CUHK-SYSU, V: ViPeR, P: PRID, G: GRID, I: i-LIDS).

|                |     Source Domains     | Target Domains | Backbone | Additional Augmentation |
|:--------------:|:----------------------:|:--------------:|:--------:|:-----------------------:|
|  Protocol (i)  |           M/D          |   D/M+V+P+G+I  | Resnet50 |     Color-Jittering     |
|  Protocol (ii) | Training set of MS+D+M |       C3       | Resnet50 |           None          |
| Protocol (iii) |         M+D+MT         |       C3       | Resnet50 |     Color-Jittering     |
|  Protocol (iv) |        M+D+C3+MT       |     V+P+G+I    | Resnet50 |     Color-Jittering     |

For Protocol (i)

|   Source   | Methods |  Avg |        | Target:Market1501 |        | Target:Duke |        | Target:PRID |        | Target:GRID |        | Target:VIPeR |        | Target:iLIDs |        |
|:----------:|---------|:----:|:------:|:-----------------:|:------:|:-----------:|:------:|:-----------:|:------:|:-----------:|:------:|:------------:|:------:|:------------:|:------:|
|            |         |  mAP | Rank-1 |        mAP        | Rank-1 |     mAP     | Rank-1 |     mAP     | Rank-1 |     mAP     | Rank-1 |      mAP     | Rank-1 |      mAP     | Rank-1 |
| Market1501 | SNR     | 50.9 |   49.6 |              84.7 |   94.4 |        33.6 |   55.1 |        42.2 |     30 |        36.7 |     29 |         42.3 |   32.3 |         65.6 |   56.7 |
|            | Ours    | **54.7** |   **53.2** |              83.5 |   92.2 |        **33.8** |   **55.5** |        **56.7** |   **44.5** |          **40** |     **31** |         **44.7** |   **35.3** |         **69.3** |   **60.7** |
| Duke       | SNR     | 51.3 |   52.2 |              33.9 |   66.7 |        72.9 |   84.4 |        45.4 |     35 |        35.3 |     26 |         41.2 |   32.6 |         79.3 |   68.7 |
|            | Ours    | **55.6** |   **56.2** |              **36.4** |   **69.2** |        72.8 |   81.7 |        **63.2** |  **53.23** |        **39.9** |   **30.4** |         **44.5** |   **34.8** |         76.7 |     68 |

For Protocol (ii)

| Target:Cuhk03 |  mAP  | Rank-1 | Rank-5 | Rank-10 |
|:-------------:|:-----:|:------:|:------:|:-------:|
|     RaMoE     |  35.5 |  36.6  |  54.3  |   64.6  |
|      Ours     | **43.77** |  **43.64** |  **65.29** |   **74.5**  |

For Protocol (iii)

| Source |      Method     | Cuhk03 |        |
|:------:|:---------------:|:------:|:------:|
|        |                 |   mAP  | Rank-1 |
| MS+D+M |     QAConv50    |   21   |  23.5  |
|        | M3L (ResNet-50) |  29.9  |  30.7  |
|        |       Ours      |  **30.86** |  **31.14** |

[1] Style Normalization and Restitution for Generalizable Person Re-Identification, in CVPR 2020.

[2] Generalizable person re-identification with a relevance-aware mixture of experts, CVPR 2021

[3] Learning to generalize unseen domains via memory-based multi-source meta-learning for person re-identiﬁcation, in CVPR 2021.

 1. Thanks for reading our theoretical derivation carefully. Equ. 6 is indeed obtained under convex conditions. Although implementing Equ.7 directly on neural networks attains poor performance, we find Equ. 6 implies an interesting phenomenon, i.e., the optimal uncertainty distribution $\mathcal{Q}$ density in KL DRO is proportional to the empirical distribution $\mathcal{P}$ density composite with the exponential term. Inspired by the observation, we reformulate KL DRO by the derivation in Equ. (8) to an importance sampling problem (Unit DRO), where the sample-wise weighted loss function in Equ. (9) will facilitate the implementation of neural networks. Thus, Unit DRO is guided by Equ.6, however, scales to neural networks well. In other words, Unit DRO is guided by the result from convex conditions, However, we find that Unit DRO works well on the highly non-convex neural network.

2.  We are very sorry for the insufficient explanation and thank you very much for your careful reading. The two questions are answered point by point as follows.
(1) Even if we can get the optimal $\tau^*$ at each training iteration, without a proper estimation for the expectation of losses over the entire training data, weights for samples are highly biased. Consider two extreme cases, if samples in one batch are all hard samples, most of them will be suppressed and if samples are all easy samples, they will be upweighted. In other words, the weighting without weight queue largely depends on the sample distribution of each batch, which is unstable, and therefore cannot bring effective performance improvement. (2) As mentioned in Section 4.3, the diagram is not a scatter diagram, the means and standard deviations are calculated from all weights of samples in 1000 training steps. Hence, there indeed are some samples with weights far from $1$. We use this diagram just to reflect that Unit DRO v2 converges stably (In contrast with constant $\tau^*$ has much more diverse weights, which is thus hard to converge. ).

3. Your suggestion is very meaningful. Model selection can be regarded as a separate research field. [1][2] designed different selection methods for DG tasks, and these methods can of course improve the performance of DG models. However, in this work, we only put forward our opinion that **using unseen test domains for model selections are unreasonable and somewhat absurd just for boosting the performance of DG-ReID models**. This, we simply choose the simplest model selection method without the use of any information from unseen test domains, as much as possible to be consistent with the previous work. We will try a variety of  model selection strategies in future work.

[1] In search of lost domain generalization. ICLR 2021.

[2] Towards a Theoretical Framework of Out-of-Distribution Generalization. NeurIPS 2021.


 1. The DGWD-ReID setting is one of our main contributions. We believe DGWD-ReID is a more challenging and reasonable ReID setting for practical applications, which is also admitted by Reviewer umsc and Reviewer eLS6. We detail the necessity as follows. 
(i) In the machine learning community, there have been already some studies [1][2] addressing the importance of not relying on domain labels. **Learning invariant feature without domain labels is becoming an important matter of concern in the ML community.** DG removes the unlabeled target data assumption from DA and we remove the requirement of domain labels of DG. Without domain labels, researchers will be forced to exploit invariant features from the training data itself, rather than the side information (manually divided domains, cameras, and frames). Thus, the proposed DGWD-ReID setting is more common and challenging.
(ii) **The domain labels, camera ids are all automatically generated and may be sub-optimal for neural networks learning invariant and generalizable ReID features**. It remains unclear how best to partition domains [3] that can benefit generalization capability most. [2] use a neural network to infer domain labels and they discover that the inferred domains are maximally informative for downstream invariant learners. How to find or learn better domain partition is a promising research branch of DGWD-ReID. Namely, not relying on these manually defined domain labels not only does not lose much information, but on the contrary, it may be more beneficial. 
(iii) The last reason is the characteristics of the ReID tasks themselves.  **ReID is research about a person and is mostly used in security systems, and naturally with higher requirements for privacy [7].**  In current DG ReID systems, each domain corresponds to a certain location (CUHK02, 03$\leftrightarrow $CUHK). Camera id corresponds to a certain view and contains more detailed location information. Frames contain temporal information. Using this information, the trace of one pedestrian is easy to monitor and predict. Hence, it is of vital importance for trying to avoid including this real information in the datasets.

2. Thank you for pointing out the question. We have reflected this concern that *the method is vulnerable to relatively large camera perspective changes* in Appendix C.5. It is indeed an innovative and well-studied problem [8] for ReID models to overcome large camera perspective challenges Second, as far as we know, current studies of DG-ReID mainly focus on the large domain divergence. The main contribution of our work also lies in this point of alleviating the effects of large domain divergence. The SOTA performance among existing DG-ReID models has proved the advantages of our methods. The observations in Fig. 11 present some limitations of our method on large camera perspective changes. However, it is not contradictory to our main contributions and may provide a possible opportunity to further boost the performance of DG-ReID. We will refer to some advanced structures or methodologies to overcome the challenge of a large camera perspective in the future.

3. We agree with you that the method is highly hyperparameter sensitive. However, we have only two hyper-parameters that is $\tau^*$ and the queue size $\mathcal{|M|}$. Finding well hyper-parameters via grid search is not hard. Besides, as shown in Table. 4, as long as $\tau^*$ does not get too small, Unit DRO generally can boost the generalization performance.

4. Firstly, many existing studies [5,6] talk about the dilemma for applying DRO to over-parameterized models. They found that, in most cases, applying DRO directly attains a result not better than empirical risk minimization (ERM). We also conduct a group of extended experiments to demonstrate *"KL DRO is not held for over-parameterized neural networks"*. Besides, we show the training of KL DRO is very unstable and attain poor performance.
|    Iter    |  3999 | 13999 | 53999 | 103999 |  Last |
|:----------:|:-----:|:-----:|:-----:|:------:|:-----:|
|   Avg_mAP  |  55.1 | 38.83 | 40.01 |  40.09 | 41.79 |
| Avg_Rank-1 | 45.93 | 28.58 | 30.25 |  30.32 | 31.84 |
5. At last, thank you again for reading the article carefully and pointing out the low-level errors in the article. We will check and correct the formulation errors and typo errors in detail in the last version.

[1] Just train twice:  Improving group robustness without training group information. ICML, 2021

[2] Environment inference for invariant learning. ICML, 2021

[3] Robustness to Spurious Correlations via Human Annotations. ICML 2020.

[5] Distributionally Robust Neural Networks for Group Shifts,ICLR, 2020

[6] Understanding deep learning requires rethinking generalization, ICLR, 2017

[7] Performance Optimization for Federated Person Re-identification via Benchmark Analysis, ACMMM, 2020

[8] Dissecting Person Re-Identification From the Viewpoint of Viewpoint, CVPR 2018.

This work provide a distributionally robust optimation (DRO) based algorithm to address the generalized person re-identification without demographics. The key difference in experimental settings is all of demographics (e.g. domain ID, camera ID) are not accessible in the training stage. To address the problem, the authors propose Unit DRO to minimize the upperbound of the expected risk on an uncertainty set (DRO algorithmic framework). The solution for intractable optimization is adopting step \tau^\star and weights queue.  PROS 1) The DGWD-ReID setting is novel, especially the privacy problem. 2) The analysis is sufficient. 3) The paper is well organized and easy to follow.

CONS 1) Although Unit DRO achieves the superior performance compared with the previous methods, it is not clear, at least theoretically, why Equ. 6 could used when convex condition is not held. 2) In Table 4, when using only step \tau^\star, the performance is close to baseline, could you discuss the reason? And in Appendix C.1, distribution diagram of step \tau^\star represents the weights are close to 1, which means all samples in the dataset are treated equally when calculating loss, it seems contradict with the motivation in the introduction part. 3) About checkpoint used on test datasets. As discussed in Section 4.2, it is unfair to adopt model selection in testing. Simply selecting the last checkpoint is acceptable, but the optimization may not stable in the training stage. In my opinion, selecting the checkpoint with the best performance on a validation dataset (different from the train and test dataset) is a better way. Overall, this paper address a novel setting for generalizable person re-identification, and the proposed method outperforms the most previous approaches. While there are some details are not clear for me.

This paper proposes a setting in person re-identification, which aims to learn domain-invariant features without using demographics information. Under this setting, this paper proposes an approach called Unit DRO by improving the existing distributionally robust optimization (DRO) method. Empirical results show the effectiveness of the proposed DRO. The strengths are as follows.

(1)	 This paper provides some theoretical analysis on the proposed DRO method.

(2)	 Extensive experiments and empirical results have been conducted to show the effectiveness of DRO.

The weaknesses and my questions are as follows.

(1)	 This paper oversells the concept of generalizable person ReID without demographics. However, this setting is not novel because it is the variant of Domain Generalizable Person Re-identification (DG ReID) that does not utilize the domain labels, camera ID and video timestamps. The existing DG ReID work [1] has not considered the above demographics and the work [2] has conducted the single-domain generalization ReID setting without domain labels. As a result, the proposed Domain Generalizable Person Re-identiﬁcation Without Demographics (DGWD) setting in this paper cannot serve as a contribution. This DGWG setting can still be included in the existing DG ReID setting.

(2)	The DRO method has been studied in the field of domain generalization, as shown in [3]. Many works about DRO have been proposed to tackle the problem of domain generalization while these key references have not been cited in this paper. This paper does not clearly clarify why DRO can help tackle the problem of DG ReID and the motivation of this paper is not clear. For another word, DRO has been utilized to tackle the problem of domain generalization and it can be directly applied to DG ReID. Thus, the proposed method is not specifically designed for DG ReID. 

(3)	From the baseline algorithms, we can see that both the cross-entropy loss and the triplet loss are used but why is the Unit DRO only enforced on the cross-entropy loss instead of the triplet loss?

(4)	Though this paper has cited the references of DG ReID [4, 5], this paper has not compared with their results on the large-scale multi-source settings proposed in [4, 5]. As a result, the existing results are not convincing to me.

(5)	Many key references about the domain generalization have not been cited. Please refer to [3, 6].


[1] Style Normalization and Restitution for Generalizable Person Re-Identification, in CVPR 2020.

[2] Domain Generalization with MixStyle, in ICLR 2021.

[3] Towards Out-Of-Distribution Generalization: A Survey, 2021.

[4] Learning to generalize unseen domains via memory-based multi-source meta-learning for person re-identiﬁcation, in CVPR 2021.

[5] Generalizable person reidentiﬁcation with relevance-aware mixture of experts, in CVPR 2021.

[6] Domain Generalization in Vision: A Survey, 2021.

 This paper oversells the concept of DG ReID without demographics, which is only a variant of DG ReID can cannot be seen as a contribution. Besides, the motivation and the necessity of solving DG ReID with the proposed Unit DRO have not clearly clarified. Many related key works have not been cited and the comparisons between other DG ReID are not enough. Based on the above reasons, I vote for " reject, not good enough".

This paper introduces an interesting task for generalizable person re-id. Specifically, it proposes to address DG re-id task without demographics. To tackle this task, the authors employ DRO and improve it by adaptively reweighting the training samples.   Pros:
1. The proposed task (DG without demographics) is practical and has not been explored. 

2. It is straightforward to employ DRO to address the proposed task, but the authors further improve KL DRO without convex conditions. The two techniques ‘step tau’ and ‘weights queue M’ are simple but effective.

3. This paper provides comprehensive experiments, including both qualitative analysis and quantitative results. 


Cons:
1. One of my confusing points is why the authors choose person re-id as the task, for me, the proposed framework is general, and it is more straightforward to consider more general domain generalization tasks, e.g., image classification. Maybe a more thorough comparison can be made with KL DRO methods in general DG tasks.

2. For DG re-id, there is another setting M3L, RaMoE [a,b]. It would be more convincing to also compare under this protocol.

[a] Learning to Generalize Unseen Domains via Memory-based Multi-Source Meta-Learning for Person Re-Identification, CVPR 2021

[b] Generalizable person re-identification with relevance-aware mixture of experts, CVPR 2021
 I vote for accepting. The proposed task is interesting and the proposed method is simple yet effective. Both numerical and qualitative results are convincing. I have two minor questions in the cons above, hopefully the authors can address my concern in the rebuttal period. 

Meta-review: This paper receives recommendations from four experts who are actively working on the Re-ID problem.  Two reviewers give more positive comments, while the comments of the remaining reviewers are relatively negative. The paper aims to achieve generalizable Re-ID without demographics (DGWD-ReID). However, two Reviewers, umsc and Cuay, pointed out that, the claimed novelty of such a problem setting is oversold, and some relevant works are not cited and compared in the original submission. More specifically, the concept of DGWD-ReID in the paper is not fully convincing, and reviewers umsc and Cuay both think DGWD-ReID setting seems to be a special case of DG-ReID, and may not need to be considered totally independently. Though the reviewer eLS6 gives more positive recommendation, an issue is still raised by this reviewer - the method is general and does not have much specific design for Re-ID, but the authors only used Re-ID datasets for evaluation. In experiments, the authors used powerful techniques/tricks in their backbone model to improve the performance. This also brings some difficulties for the readers to assess which part is really working in their method. It will be good if authors can further improve the paper.

Decision: Reject