Gradients before clip are much lager than the clip bound

Zark: <p>Hi,</p>
<p>I’m using Opacus to implement DPSGD in my program. One thing I noticed is that the gradients to be clipped are much larger than the <code>max_grad_norm</code>, as the screenshot from the console output.</p>
<p>The code is from the opacus tutorial <a href="https://github.com/pytorch/opacus/blob/main/tutorials/building_image_classifier.ipynb" class="inline-onebox" rel="noopener nofollow ugc">opacus/tutorials/building_image_classifier.ipynb at main · pytorch/opacus · GitHub</a> I only modified the <code>MAX_PHYSICAL_BATCH_SIZE </code> to fit in my machine.  I print the <code>per_sample_norms</code> in the <code>clip_and_accumulate</code> function in the <code>DPOptimizer</code> class of Opacus by following code,</p>
<pre><code class="lang-auto">    def clip_and_accumulate(self):
        per_param_norms = [g.view(len(g), -1).norm(2, dim=-1) for g in self.grad_samples]
        per_sample_norms = torch.stack(per_param_norms, dim=1).norm(2, dim=1)
        per_sample_clip_factor = (self.max_grad_norm / (per_sample_norms + 1e-6)).clamp(
            max=1.0 )

        print(f'80% gradient norms are less than {np.percentile(per_sample_norms.cpu(), 80)}')

        for p in self.params:
            _check_processed_flag(p.grad_sample)
            grad_sample = _get_flat_grad_sample(p)
            grad = torch.einsum("i,i...", per_sample_clip_factor, grad_sample)

            if p.summed_grad is not None:
                p.summed_grad += grad
            else:
                p.summed_grad = grad

            _mark_as_processed(p.grad_sample)

</code></pre>
<p>And get the following results, the <code>per_sample_norms</code> are much larger than the clip bound, which is C=1.2.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/0/402e91fd5a6ad84037e77b9c979e381a857391f3.png" data-download-href="https://discuss.pytorch.org/uploads/default/402e91fd5a6ad84037e77b9c979e381a857391f3" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/4/0/402e91fd5a6ad84037e77b9c979e381a857391f3_2_312x374.png" alt="image" data-base62-sha1="99MkJEmLKXyBHN4ApbLhrC7CD2r" width="312" height="374" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/4/0/402e91fd5a6ad84037e77b9c979e381a857391f3_2_312x374.png, https://discuss.pytorch.org/uploads/default/optimized/3X/4/0/402e91fd5a6ad84037e77b9c979e381a857391f3_2_468x561.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/4/0/402e91fd5a6ad84037e77b9c979e381a857391f3_2_624x748.png 2x" data-dominant-color="E9E9E9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">690×829 116 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Can somebody tell me why? This is strange because the gradient before the clip is much larger than the clip bound.</p>
<p>Thanks in advance</p>

ashkan_software: <p>Hi Zark,</p>
<p>I believe this is the correct behavior. The gradients before clipping can have any magnitude; it’s the responsibility of the clipping in DP-SGD to bring the gradients in the acceptable range. So, gradients before clipping are in arbitrary ranges. Once we clip them, they get smaller.</p>
<p>Does that make sense?</p>

Zark: <p>Thanks, Got it. And I’m wondering what makes a good choice of clip bound, maybe we should check the distribution of gradients and then select clip by percentage? like 80%</p>

ashkan_software: <p>Hi <a class="mention" href="/u/zark">@Zark</a></p>
<p>Great question. As far as I know, clip bound is a hyper parameter and we should treat it just like another hyper parameter, such as learning rate. The recent paper <a href="https://arxiv.org/pdf/2204.13650.pdf" rel="noopener nofollow ugc">https://arxiv.org/pdf/2204.13650.pdf</a> uses 0.1, 1, and 10 (Table 7).  This paper also sheds light on choice of clip bound, how to do image classification with DP, so I think this is very relevant to your use case.</p>
<p>Best of luck</p>

Zark: <p>Hi ashkan,</p>
<p>thanks for sharing this paper. I’ll have a look at it <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

ffuuugor: <p>One more relevant paper to look at is <a href="https://arxiv.org/pdf/2110.05679.pdf" rel="noopener nofollow ugc">Large Language Models Can Be Strong Differentially Private Learners</a>. Authors explore wide variety of hyper-parameters for large language model fune-tuning task, including clipping norm (section 3.1.2)</p>
<p>tl;dr - clippig most (if not all gradients) might even be the desired behaviour, as it maximizes signal-to-noise ratio. Clipping itself shouldn’t have major effect on the model training, as the scale of the weights in most layers doesn’t matter and the model will adapt to the new scale as it trains</p>