Cast BatchNorm2d to int32

username: shabashaash
cooked: <p>I have several BatchNorm2d layers without some additional params. As I know the webgl runtime for onnxruntime-web doesnt support int64. But tracked parametrs for BatchNorm2d are saved in int64 (because of possible overflow of int32). I tried multiple ways to convert this several params to int32 or even float32 (I know its bad, but I just tried) and they stayed the same. Maybe its because of original implementation of BatchNorm which doesnt allow to change the type.</p>
<p>So as mentioned here (<a href="https://github.com/pytorch/pytorch/issues/14807" rel="noopener nofollow ugc">pytorch/pytorch#14807</a>) there are multiple ways to solve this.</p>
<p>What have I tried so far:<br>
1.Method like here (<a href="https://github.com/ultralytics/yolov5/issues/250" rel="noopener nofollow ugc">ultralytics/yolov5#250</a>) using setattr()<br>
2.1 By adding .to(torch.int32) - which returned “nn.Module.to only accepts floating point or complex dtypes, but got desired dtype=torch.int32” which is understandable.<br>
2.2 By adding .to(torch.float32) - which doesnt change anything, params stayed INT64.<br>
Example of second approach:</p>
<pre><code class="lang-auto">self.up3 = nn.Sequential( nn.Upsample(scale_factor=2, mode='bilinear'), nn.Conv2d(512, 256, kernel_size=3, stride=1, padding=1), nn.BatchNorm2d(256).to(torch.float32), activation )
</code></pre>
<ol start="3">
<li>I tried this script - onnx-typecast (search on github) - no luck<br>
Params in INT64:</li>
</ol>
<pre><code class="lang-auto">['first_layer.2.num_batches_tracked', 'down0.1.num_batches_tracked', 'down1.1.num_batches_tracked', 'down2.1.num_batches_tracked', 'down3.1.num_batches_tracked', 'up3.2.num_batches_tracked', 'up2.2.num_batches_tracked', 'up1.2.num_batches_tracked', 'up0.2.num_batches_tracked']
</code></pre>
<p>Also Im loading pretrained model (not training from scratch).</p>

username: anantguptadbl
cooked: <p><a class="mention" href="/u/shabashaash">@shabashaash</a> This looks like a night time endless coffee problem. Do you have the onnx file/model which we can play with</p>

username: shabashaash
cooked: <p>Sure. I already trying to solve it for 3 days straight. The model is (<a href="https://github.com/neuralchen/SimSwap" class="inline-onebox" rel="noopener nofollow ugc">GitHub - neuralchen/SimSwap: An arbitrary face-swapping framework on images and videos with one single trained model!</a>). Im using the pretrained 512HQ model. (Actual model is in fs_networks_512 named Generator_Adain_Upsample).<br>
Here is <a href="https://drive.google.com/drive/folders/1exI5G4KLJhcAUI2WVSfPj_9-gsdvyj17?usp=sharing" rel="noopener nofollow ugc">link</a> to folder on google drive with 2 models:<br>
true_visual_512_opset13_nc_float32cast.onnx - just added .to(‘float32’)<br>
true_visual_512_opset13_nc_float32cast_int32.onnx - added .to(‘float32’) and also used onnx-typecast script.</p>
<p>Also when im trying to run model on onnx-runtime (Python) which was converted with onnx-typecast script, Im getting this error:<br>
“[ONNXRuntimeError] : 10 : INVALID_GRAPH : Load model from /content/onnx_models/true_visual_512_opset13_nc_float32cast_int32.onnx failed:This is an invalid model. Type Error: Type ‘tensor(int32)’ of input parameter (219) of operator (Unsqueeze) in node (Unsqueeze_76) is invalid.”</p>
<p>UPD:<br>
And with same model on onnxruntime-web(nodeJS) with webgl (works well on wasm = cpu, but too long):<br>
“failed to inference ONNX model: Error: unrecognized input ‘’ for node: Resize_1242.”. I suppose its because some of params are not converted to INT32 properly.<br>
Resize block (have 4 of them):<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/6/b6b92d7814bd3da0f1a761ebab99c0e396bad8ed.png" data-download-href="https://discuss.pytorch.org/uploads/default/b6b92d7814bd3da0f1a761ebab99c0e396bad8ed" title="изображение"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/b/6/b6b92d7814bd3da0f1a761ebab99c0e396bad8ed_2_492x500.png" alt="изображение" data-base62-sha1="q4rDJlurEYhScRRNo9fc2Wce35r" width="492" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/b/6/b6b92d7814bd3da0f1a761ebab99c0e396bad8ed_2_492x500.png, https://discuss.pytorch.org/uploads/default/original/3X/b/6/b6b92d7814bd3da0f1a761ebab99c0e396bad8ed.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/b/6/b6b92d7814bd3da0f1a761ebab99c0e396bad8ed.png 2x" data-dominant-color="F4F4F4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">изображение</span><span class="informations">720×731 26.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>