Niceties of model on conv2d (work with images)

username: P3TU5H0K
cooked: <p>Well… I have 2 global problems , may be I have other problems, but I didnt found they and it’s not SO important (except optimization).<br>
I tried to train models like: base upscale (2x/4x), remove JPG defects.</p>
<p>(**All details of code at the bottom. **)</p>
<p><strong>Problem 1:</strong><br>
After I trained model and test it, output looks normal in general, BUT it have artifacts on edges of image.<br>
Example (input/output/target - from tests):<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/8/c/8cf154728a49a8bf50b18c15d1764fdd9f95747b.png" alt="image_№9bbdd52eb441fa230949b670d2893e90324b8308r1-1920-1080v2_uhq.jpg_0" data-base62-sha1="k6PTlEHIsLMEp28Qv5uU69MI4H1" width="200" height="200">  <div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/e/ce7a63d525a17c63671b49789a7573823b2127ce.png" data-download-href="https://discuss.pytorch.org/uploads/default/ce7a63d525a17c63671b49789a7573823b2127ce" title="image_№9bbdd52eb441fa230949b670d2893e90324b8308r1-1920-1080v2_uhq.jpg_1"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/c/e/ce7a63d525a17c63671b49789a7573823b2127ce_2_500x500.png" alt="image_№9bbdd52eb441fa230949b670d2893e90324b8308r1-1920-1080v2_uhq.jpg_1" data-base62-sha1="tsAzQF7YOOGUAVMDT3qA4Jc6O4u" width="500" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/c/e/ce7a63d525a17c63671b49789a7573823b2127ce_2_500x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/c/e/ce7a63d525a17c63671b49789a7573823b2127ce_2_750x750.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/c/e/ce7a63d525a17c63671b49789a7573823b2127ce.png 2x" data-dominant-color="897574"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image_№9bbdd52eb441fa230949b670d2893e90324b8308r1-1920-1080v2_uhq.jpg_1</span><span class="informations">800×800 812 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div> <div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/a/4/a4641ae12a9a9229b3ac35df18235e213a3b7440.png" data-download-href="https://discuss.pytorch.org/uploads/default/a4641ae12a9a9229b3ac35df18235e213a3b7440" title="image_№9bbdd52eb441fa230949b670d2893e90324b8308r1-1920-1080v2_uhq.jpg_2"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/a/4/a4641ae12a9a9229b3ac35df18235e213a3b7440_2_500x500.png" alt="image_№9bbdd52eb441fa230949b670d2893e90324b8308r1-1920-1080v2_uhq.jpg_2" data-base62-sha1="nsgMiWTuvOgIFdf5SKfoVap26Yw" width="500" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/a/4/a4641ae12a9a9229b3ac35df18235e213a3b7440_2_500x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/a/4/a4641ae12a9a9229b3ac35df18235e213a3b7440_2_750x750.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/a/4/a4641ae12a9a9229b3ac35df18235e213a3b7440.png 2x" data-dominant-color="8B747B"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image_№9bbdd52eb441fa230949b670d2893e90324b8308r1-1920-1080v2_uhq.jpg_2</span><span class="informations">800×800 824 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>This artfacts small: ~ 5 - 20 px, and I could leave this, but if I don’t upscale whole image (I cant do this with big img like full hd to 4k), I upscale parts of image. So, in this situation this small artifact is more important.<br>
Example:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/9/996d4fb2adf8ef4c6a83ef446c72ca18fc05ab94.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/996d4fb2adf8ef4c6a83ef446c72ca18fc05ab94" title="9bbdd52eb441fa230949b670d2893e90324b8308r1-1920-1080v2_uhq"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/9/9/996d4fb2adf8ef4c6a83ef446c72ca18fc05ab94_2_690x388.jpeg" alt="9bbdd52eb441fa230949b670d2893e90324b8308r1-1920-1080v2_uhq" data-base62-sha1="lThfAo9IFEIImjeGJCilYCDuc8Q" width="690" height="388" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/9/9/996d4fb2adf8ef4c6a83ef446c72ca18fc05ab94_2_690x388.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/9/9/996d4fb2adf8ef4c6a83ef446c72ca18fc05ab94_2_1035x582.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/9/9/996d4fb2adf8ef4c6a83ef446c72ca18fc05ab94_2_1380x776.jpeg 2x" data-dominant-color="7E6B70"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">9bbdd52eb441fa230949b670d2893e90324b8308r1-1920-1080v2_uhq</span><span class="informations">1920×1080 787 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p><strong>Problem 2:</strong><br>
I have artifacts looks like gauss noise. I think that it is because my dataset is not best, I made it by myself, and it has not many images with some colors (so artifacts occur on images which have seldom colors). But may be it haw artifacts by another thing, therefore mentioned this.<br>
<em>Now I have my dataset (100 000 images) + another dataset (350 000 images), but I don’t use them all always, because it is too long</em></p>
<pre><code> Example (model output/target with gauss noise) - it's after 100 000 images with 5 epochs:
</code></pre>
<p><em>target is bigger beacuse I resize inputs for the model, so that give max 800x800 image (other OOM error)</em></p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/b/4b46717bd7724e06e9f519c7ade182d4f860cb1c.png" data-download-href="https://discuss.pytorch.org/uploads/default/4b46717bd7724e06e9f519c7ade182d4f860cb1c" title="image_№image_№96-965159_wallpaper-anime-hot-anime-girl-overlord-voracity-by..jpg_2.png_1"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/4/b/4b46717bd7724e06e9f519c7ade182d4f860cb1c_2_690x386.png" alt="image_№image_№96-965159_wallpaper-anime-hot-anime-girl-overlord-voracity-by..jpg_2.png_1" data-base62-sha1="aJUJwPXClKuq00JjJqiB5yXUndG" width="690" height="386" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/4/b/4b46717bd7724e06e9f519c7ade182d4f860cb1c_2_690x386.png, https://discuss.pytorch.org/uploads/default/original/3X/4/b/4b46717bd7724e06e9f519c7ade182d4f860cb1c.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/4/b/4b46717bd7724e06e9f519c7ade182d4f860cb1c.png 2x" data-dominant-color="69473D"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image_№image_№96-965159_wallpaper-anime-hot-anime-girl-overlord-voracity-by..jpg_2.png_1</span><span class="informations">800×448 518 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div> <div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/6/469a6252c2f343f61b9d1ace2cf72e3cd49f58c4.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/469a6252c2f343f61b9d1ace2cf72e3cd49f58c4" title="96-965159_wallpaper-anime-hot-anime-girl-overlord-voracity-by."><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/4/6/469a6252c2f343f61b9d1ace2cf72e3cd49f58c4_2_690x388.jpeg" alt="96-965159_wallpaper-anime-hot-anime-girl-overlord-voracity-by." data-base62-sha1="a4AbJCtP3BZpQfDtd9KipfFSJIE" width="690" height="388" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/4/6/469a6252c2f343f61b9d1ace2cf72e3cd49f58c4_2_690x388.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/4/6/469a6252c2f343f61b9d1ace2cf72e3cd49f58c4_2_1035x582.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/4/6/469a6252c2f343f61b9d1ace2cf72e3cd49f58c4_2_1380x776.jpeg 2x" data-dominant-color="674B3D"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">96-965159_wallpaper-anime-hot-anime-girl-overlord-voracity-by.</span><span class="informations">1920×1080 315 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/0/8/08d08f4cf5e48497abf5547e47a17499b0f9ae93.png" data-download-href="https://discuss.pytorch.org/uploads/default/08d08f4cf5e48497abf5547e47a17499b0f9ae93" title="image_№tof3.png_1"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/0/8/08d08f4cf5e48497abf5547e47a17499b0f9ae93_2_500x500.png" alt="image_№tof3.png_1" data-base62-sha1="1fYEJwTWamdX1CFFYk41kT7nK6f" width="500" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/0/8/08d08f4cf5e48497abf5547e47a17499b0f9ae93_2_500x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/0/8/08d08f4cf5e48497abf5547e47a17499b0f9ae93_2_750x750.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/0/8/08d08f4cf5e48497abf5547e47a17499b0f9ae93.png 2x" data-dominant-color="5B4346"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image_№tof3.png_1</span><span class="informations">800×800 762 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div> <div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/f/1ff19e133e7b5042d8c84f087383985f5df2412d.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/1ff19e133e7b5042d8c84f087383985f5df2412d" title="tof3"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/f/1ff19e133e7b5042d8c84f087383985f5df2412d_2_690x429.jpeg" alt="tof3" data-base62-sha1="4yAsO73ZULJPKZ7g02xOjUPcSnP" width="690" height="429" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/f/1ff19e133e7b5042d8c84f087383985f5df2412d_2_690x429.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/1/f/1ff19e133e7b5042d8c84f087383985f5df2412d_2_1035x643.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/1/f/1ff19e133e7b5042d8c84f087383985f5df2412d_2_1380x858.jpeg 2x" data-dominant-color="5F484B"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">tof3</span><span class="informations">1920×1195 1.03 MB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p><strong>My question for both problems:</strong> Can I fix it and how? I just need to have more training, or something else? Or I have mistakes on layers, or preparing my dataset? I can’t check them all by myself because the processing will take 30 days on my computer with my code. XD</p>
<p><strong>IMPORTANT:</strong> If you saw such a discussion with the same question and decision, you don’t have to explain it to me, just give the link to that, and I’ll check it.<br>
And if you see some mistakes with optimization (I know - they are), please indicate this if it’s not difficult for you. It’s very important for me, because… Come on, 30 DAYS…</p>
<p><strong>My code</strong><br>
“ngf” I always do 64.</p>
<p>Model (example for 4x).<br>
Model’s layers always looks like :</p>
<pre><code class="lang-auto">self.g0 = nn.Sequential(
            #nn.Dropout2d(p=0.2),
            nn.Conv2d(3, ngf * 4, 3, stride=1, padding=1, bias=False),
            nn.BatchNorm2d(ngf * 4),
            nn.ReLU(True),
        )
        
        
        self.g1 = nn.Sequential(
            nn.Conv2d(ngf * 4, ngf * 4, 3, stride=1, padding=1, bias=False),
            nn.BatchNorm2d(ngf * 4),
            nn.ReLU(True),
            
            nn.Conv2d(ngf * 4, ngf * 4, 3, stride=1, padding=1, bias=False),
            nn.BatchNorm2d(ngf * 4),
            nn.ReLU(True)
        )
        
        self.main1 = nn.Sequential(
            nn.ConvTranspose2d(ngf * 4, ngf * 4, 4, stride=2, padding=1, bias=False),
            nn.BatchNorm2d(ngf * 4),
            nn.ReLU(True),
        )

        self.g2 = nn.Sequential(
            nn.Conv2d(ngf * 4, ngf * 4, 3, stride=1, padding=1, bias=False),
            nn.BatchNorm2d(ngf * 2),
            nn.ReLU(True),
            
            nn.Conv2d(ngf * 4, ngf * 4, 3, stride=1, padding=1, bias=False),
            nn.BatchNorm2d(ngf * 4),
            nn.ReLU(True),
        )
        
        
            
        self.main2 = nn.Sequential(            
            nn.ConvTranspose2d(ngf * 4, ngf, 4, stride=2, padding=1, bias=False),
            nn.BatchNorm2d(ngf),
            nn.ReLU(True),

            nn.Conv2d(ngf, 3, 3, stride=1, padding=1, bias=False),
            nn.Tanh()   
        )
</code></pre>
<p>Forward function like this:</p>
<pre><code class="lang-auto">def forward(self, X):
        X = self.g0(X)

        X = self.g1(X) + X
        X = self.main1(X)

        X = self.g2(X) + X
        X = self.main2(X)
        return X
</code></pre>
<p>And training step:</p>
<pre><code class="lang-auto">def training_step(self, batch, batch_idx):
        
        x, _ = batch
        x = x.view(self.hparams['batch_size'], x.size()[-3], x.size()[-2], x.size()[-1]).to(self.device)
        
        #x = F.interpolate(x, size=(int(y[0]), int(y[1])), mode='bicubic', align_corners=False)

        g = self.generator(F.interpolate(x, size=(x.size()[-2] // self.increase, x.size()[-1] // self.increase), mode='bicubic', align_corners=False))
        #########################____GRADS____############################
        self.generator.zero_grad()

        g_real_loss = criterion(g, F.interpolate(x, size=(x.size()[-2] // self.increase * self.increase, x.size()[-1] // self.increase * self.increase), mode='bicubic', align_corners=False))
        g_real_loss.backward(retain_graph=True)

        self.opt_g.step()
        ###################################################################
</code></pre>
<p>Exaple for “remove defects”:</p>
<pre><code>  Model's layers:
</code></pre>
<pre><code class="lang-auto">self.ngf = ngf
        
        self.g0 = nn.Sequential(
            #nn.Dropout2d(p=0.2),
            nn.Conv2d(3, ngf*4, 3, stride=1, padding=1, bias=False),
            nn.BatchNorm2d(ngf*4),
            nn.ReLU(inplace=False),
        )
        
        
        self.g1 = nn.Sequential(
            nn.Conv2d(ngf * 4, ngf * 4, 3, stride=1, padding=1, bias=False),
            nn.BatchNorm2d(ngf * 4),
            nn.ReLU(inplace=False),
            
            nn.Conv2d(ngf * 4, ngf *4, 3, stride=1, padding=1, bias=False),
            nn.BatchNorm2d(ngf * 4),
            nn.ReLU(inplace=False)
        )
        
        self.main1 = nn.Sequential(
            nn.Conv2d(ngf * 4, ngf * 4, 3, stride=1, padding=1, bias=False),
            nn.BatchNorm2d(ngf * 4),
            nn.ReLU(inplace=False),
            # state size. (ngf*8) x 4 x 4
        )

        self.g2 = nn.Sequential(
            nn.Conv2d(ngf * 4, ngf * 4, 3, stride=1, padding=1, bias=False),
            nn.BatchNorm2d(ngf * 4),
            nn.ReLU(inplace=False),
            
            nn.Conv2d(ngf * 4, ngf * 4, 3, stride=1, padding=1, bias=False),
            nn.BatchNorm2d(ngf * 4),
            nn.ReLU(inplace=False),
        )
        
            
        self.main2 = nn.Sequential(            
            nn.Conv2d(ngf * 4, ngf*4, 3, stride=1, padding=1, bias=False),
            nn.BatchNorm2d(ngf*4),
            nn.ReLU(inplace=False),

            nn.Conv2d(ngf*4, 3, 3, stride=1, padding=1, bias=False),
            nn.Tanh() 
        )
</code></pre>
<p>And train step, here I have 3 versions of this step, where I just train whole resized image,<br>
where I cut image and train with these parts, or I cuts image, put parts into forward, but loss get by criterion of all of parts and whole target image (I cuts it double, because first cut for exclude OOM error, and next to train by parts). I choose this, because if I do this, it dont require many resources in forward.<br>
<em>I commented parts of old versions step</em></p>
<pre><code class="lang-auto">def training_step_crop(self, batch, batch_idx):
        x, y = batch

        big_res = max(round(x.size()[-2] / self.hparams['max_input'][0] + 0.5), round(x.size()[-1] / self.hparams['max_input'][1] + 0.5))

        for r0 in range(big_res):
            for r1 in range(big_res):
                big_tmp_x = torch.chunk(x,big_res,dim=-2)[r0]
                big_tmp_x = torch.chunk(big_tmp_x,big_res,dim=-1)[r1]

                big_tmp_y = torch.chunk(y,big_res,dim=-2)[r0]
                big_tmp_y = torch.chunk(big_tmp_y,big_res,dim=-1)[r1]

                res = max(round(big_tmp_x.size()[-2] / self.hparams['max_crop'][0] + 0.5), round(big_tmp_x.size()[-1] / self.hparams['max_crop'][1] + 0.5))
        
                outs = []
                for it0 in range(res):
                    out0 = []
                    for it1 in range(res):
                        tmp_x = torch.chunk(big_tmp_x,res,dim=-2)[it0]
                        tmp_x = torch.chunk(tmp_x,res,dim=-1)[it1]
                        #tmp_y = torch.chunk(y,res,dim=-2)[it0]
                        #tmp_y = torch.chunk(tmp_y,res,dim=-1)[it1]
            


                        #print('Memory:', get_gpu_memory_map()) - I chaked memory...

                        tmp_x = tmp_x.view(1, 3, tmp_x.size()[-2], tmp_x.size()[-1]).to(self.device)
                        #tmp_y = tmp_y.view(1, 3, tmp_y.size()[-2], tmp_y.size()[-1]).to(self.device)
                        tmp_x = self.generator(tmp_x)
                        out0.append(tmp_x.cpu())
                        #########################__GENERATOR__#############################
                        #self.generator.zero_grad()
        
                        #loss = self.criterion(tmp_x, tmp_y)
                        #loss.backward(retain_graph=True)
                

                        #self.opt.step()
                        ###################################################################
                        del tmp_x
                        #del tmp_y
                

                    outs.append(torch.cat(out0, 3))
                    del out0
                outs = torch.cat(outs, 2)#.clone()
                #####################################
                self.generator.zero_grad()

                loss = self.criterion(outs.cpu(),big_tmp_y)
                loss.backward()

                self.opt.step()
                #####################################
                del outs
                torch.cuda.empty_cache()
                #with torch.cuda.device('cuda'):
                #torch.cuda.empty_cache()
</code></pre>
<p>My dataset (~same for both models):<br>
“RandomPilTransforms” - is my class of same flips and deformations for sequential of images (so, here it useless, but still…)<br>
I use save image with small quality on JPG and load it again, for simulation JPG defects, because I didn’t found function, which do this without save. If you know same functions, I’ll be glad to hear about it.</p>
<pre><code class="lang-auto">class MyDataset(Dataset):
	def __init__(self, path, input_size, quality=50, max_len=False, save_path=r'data\cash'):

		RPTargs = {
		'perspective': {
		'deformation': 0.5,
		'chance': 0.5,
		'resample': Image.BICUBIC,
		'fill': 0,
		'fillcolor': None
		},
		'flip_horizontal': 0.5,
		'flip_vertical': 0.5,
		'rotate_right': 0.5,
		'rotate_left': 0.5
		}
		self.transform = RandomPilTransforms(**RPTargs)

		
		self.path = path
		self.quality = quality
		self.save_path = save_path
		
		
		self.piltotensor = transforms.ToTensor()

		self.input_size = input_size

		self.names = []
		for dirpath,_,filenames in os.walk(path):
			for f in filenames:
				self.names.append(os.path.abspath(os.path.join(dirpath, f)))

		random.shuffle(self.names, random.seed())
		print('')
		print('All dataset:', len(self.names))


		
		
		self.len = len(self.names)
		if max_len:
			self.len = min(max_len, self.len)
		print('Using dataset:', self.len)
		print('')	
		

	def __getitem__(self, index):
		try:
			y = Image.open(self.names[index]).convert('RGB')
		except:
			del self.names[index]
			return self.__getitem__(index)

		if y.size[0] * y.size[1] &gt; self.input_size[0] * self.input_size[1]:
			if y.size[0] &gt; y.size[1]:
				y = y.resize((self.input_size[0], self.input_size[0] * y.size[1] // y.size[0]))
			else:
				y = y.resize((self.input_size[1] * y.size[0] // y.size[1], self.input_size[1]))
				
		y = self.transform([y])[0]
		x = self.noisy(y)
		
		#y = cv2.convertScaleAbs(np.asarray(y))
		x = self.piltotensor(x)
		y = self.piltotensor(y)
		
		return x, y
	
	def noisy(self, image):
		image.save(self.save_path  + r'\cash_img.jpg', quality=self.quality)
		image = Image.open(self.save_path  + r'\cash_img.jpg').convert('RGB')
		return image


	def __len__(self):
		return self.len




(**All details of code at the bottom. **)

**Problem 1:**
     After I trained model and test it, output looks normal in general, BUT it have artifacts on edges of image.
     Example (input/output/target - from tests):
![image_№9bbdd52eb441fa230949b670d2893e90324b8308r1-1920-1080v2_uhq.jpg_0|200x200](upload://k6PTlEHIsLMEp28Qv5uU69MI4H1.png)  ![image_№9bbdd52eb441fa230949b670d2893e90324b8308r1-1920-1080v2_uhq.jpg_1|500x500](upload://tsAzQF7YOOGUAVMDT3qA4Jc6O4u.png) ![image_№9bbdd52eb441fa230949b670d2893e90324b8308r1-1920-1080v2_uhq.jpg_2|500x500](upload://nsgMiWTuvOgIFdf5SKfoVap26Yw.png) 

     This artfacts small: ~ 5 - 20 px, and I could leave this, but if I don't upscale whole image (I cant do this with big img like full hd to 4k), I upscale parts of image. So, in this situation this small artifact is more important.
     Exaple:
![9bbdd52eb441fa230949b670d2893e90324b8308r1-1920-1080v2_uhq|690x388](upload://lThfAo9IFEIImjeGJCilYCDuc8Q.jpeg) 

**Problem 2:** 
     I have artifacts looks like gauss noise. I thinks that it because my dataset is not best, I made it myself, and he have not many images with some colors (so artifacts meets on images which have seldom colors). But may be it have artifacts by another thing, therefore mentioned this. 
*Now I have my dataset (100 000 images) + another dataset (350 000 images), but not use this always, because it is too long*

     Example (model output/target with gauss noise) - it's after 100 000 images with 5 epochs:
*target is bigger beacuse I resize inputs for the model, so that give max 800x800 image (other OOM error)*

![image_№image_№96-965159_wallpaper-anime-hot-anime-girl-overlord-voracity-by..jpg_2.png_1|690x386](upload://aJUJwPXClKuq00JjJqiB5yXUndG.png) ![96-965159_wallpaper-anime-hot-anime-girl-overlord-voracity-by.|690x388](upload://a4AbJCtP3BZpQfDtd9KipfFSJIE.jpeg) 

![image_№tof3.png_1|500x500](upload://1fYEJwTWamdX1CFFYk41kT7nK6f.png) ![tof3|690x429](upload://4yAsO73ZULJPKZ7g02xOjUPcSnP.jpeg) 



**My question for both problems:** Can I fix it and how? I just need to have more training, or something else? Or I have mistakes on layers, or preparing my dataset? I can't check it myself because on my code with my computer all dataset runs 30 days. XD


**IMPORTANT:** If you saw discuss with same question and with decision, you don't have to explain it to me, just give the link to that, and I'll check it.
And if you see some mistakes with optimization (I know - they are), please indicate this if it's not difficult for you. It very important for me, because... Come on, 30 DAYS...


**My code**
"ngf" I always do 64.

Model (example for 4x).
     Model's layers always looks like :

</code></pre>
<p>self.g0 = nn.Sequential(<br>
<span class="hashtag-raw">#nn</span>.Dropout2d(p=0.2),<br>
nn.Conv2d(3, ngf * 4, 3, stride=1, padding=1, bias=False),<br>
nn.BatchNorm2d(ngf * 4),<br>
nn.ReLU(True),<br>
)</p>
<pre><code>    self.g1 = nn.Sequential(
        nn.Conv2d(ngf * 4, ngf * 4, 3, stride=1, padding=1, bias=False),
        nn.BatchNorm2d(ngf * 4),
        nn.ReLU(True),
        
        nn.Conv2d(ngf * 4, ngf * 4, 3, stride=1, padding=1, bias=False),
        nn.BatchNorm2d(ngf * 4),
        nn.ReLU(True)
    )
    
    self.main1 = nn.Sequential(
        nn.ConvTranspose2d(ngf * 4, ngf * 4, 4, stride=2, padding=1, bias=False),
        nn.BatchNorm2d(ngf * 4),
        nn.ReLU(True),
    )

    self.g2 = nn.Sequential(
        nn.Conv2d(ngf * 4, ngf * 4, 3, stride=1, padding=1, bias=False),
        nn.BatchNorm2d(ngf * 2),
        nn.ReLU(True),
        
        nn.Conv2d(ngf * 4, ngf * 4, 3, stride=1, padding=1, bias=False),
        nn.BatchNorm2d(ngf * 4),
        nn.ReLU(True),
    )
    
    
        
    self.main2 = nn.Sequential(            
        nn.ConvTranspose2d(ngf * 4, ngf, 4, stride=2, padding=1, bias=False),
        nn.BatchNorm2d(ngf),
        nn.ReLU(True),

        nn.Conv2d(ngf, 3, 3, stride=1, padding=1, bias=False),
        nn.Tanh()   
    )
</code></pre>
<pre><code class="lang-auto">Forward function like this:
</code></pre>
<p>def forward(self, X):<br>
X = self.g0(X)</p>
<pre><code>    X = self.g1(X) + X
    X = self.main1(X)

    X = self.g2(X) + X
    X = self.main2(X)
    return X
</code></pre>
<pre><code class="lang-auto">And training step:
</code></pre>
<p>def training_step(self, batch, batch_idx):</p>
<pre><code>    x, _ = batch
    x = x.view(self.hparams['batch_size'], x.size()[-3], x.size()[-2], x.size()[-1]).to(self.device)
    
    #x = F.interpolate(x, size=(int(y[0]), int(y[1])), mode='bicubic', align_corners=False)

    g = self.generator(F.interpolate(x, size=(x.size()[-2] // self.increase, x.size()[-1] // self.increase), mode='bicubic', align_corners=False))
    #########################____GRADS____############################
    self.generator.zero_grad()

    g_real_loss = criterion(g, F.interpolate(x, size=(x.size()[-2] // self.increase * self.increase, x.size()[-1] // self.increase * self.increase), mode='bicubic', align_corners=False))
    g_real_loss.backward(retain_graph=True)

    self.opt_g.step()
    ###################################################################
</code></pre>
<pre><code class="lang-auto">



Exaple for "remove defects":

      Model's layers:
</code></pre>
<p>self.ngf = ngf</p>
<pre><code>    self.g0 = nn.Sequential(
        #nn.Dropout2d(p=0.2),
        nn.Conv2d(3, ngf*4, 3, stride=1, padding=1, bias=False),
        nn.BatchNorm2d(ngf*4),
        nn.ReLU(inplace=False),
    )
    
    
    self.g1 = nn.Sequential(
        nn.Conv2d(ngf * 4, ngf * 4, 3, stride=1, padding=1, bias=False),
        nn.BatchNorm2d(ngf * 4),
        nn.ReLU(inplace=False),
        
        nn.Conv2d(ngf * 4, ngf *4, 3, stride=1, padding=1, bias=False),
        nn.BatchNorm2d(ngf * 4),
        nn.ReLU(inplace=False)
    )
    
    self.main1 = nn.Sequential(
        nn.Conv2d(ngf * 4, ngf * 4, 3, stride=1, padding=1, bias=False),
        nn.BatchNorm2d(ngf * 4),
        nn.ReLU(inplace=False),
        # state size. (ngf*8) x 4 x 4
    )

    self.g2 = nn.Sequential(
        nn.Conv2d(ngf * 4, ngf * 4, 3, stride=1, padding=1, bias=False),
        nn.BatchNorm2d(ngf * 4),
        nn.ReLU(inplace=False),
        
        nn.Conv2d(ngf * 4, ngf * 4, 3, stride=1, padding=1, bias=False),
        nn.BatchNorm2d(ngf * 4),
        nn.ReLU(inplace=False),
    )
    
        
    self.main2 = nn.Sequential(            
        nn.Conv2d(ngf * 4, ngf*4, 3, stride=1, padding=1, bias=False),
        nn.BatchNorm2d(ngf*4),
        nn.ReLU(inplace=False),

        nn.Conv2d(ngf*4, 3, 3, stride=1, padding=1, bias=False),
        nn.Tanh() 
    )
</code></pre>
<pre><code class="lang-auto">
And training step, here I have 3 versions of this step, where I just train whole resized image,
where I cut image and train by these parts, or I cut image, put these parts into forward, but get loss by criterion of all parts and whole target image (I cut it double, because firstly I cut for exclude OOM error, and then train by parts). I choose this way to reduce resources consumption by forward function.
*I commented parts of old step versions*
</code></pre>
<p>def training_step_crop(self, batch, batch_idx):<br>
x, y = batch</p>
<pre><code>    big_res = max(round(x.size()[-2] / self.hparams['max_input'][0] + 0.5), round(x.size()[-1] / self.hparams['max_input'][1] + 0.5))

    for r0 in range(big_res):
        for r1 in range(big_res):
            big_tmp_x = torch.chunk(x,big_res,dim=-2)[r0]
            big_tmp_x = torch.chunk(big_tmp_x,big_res,dim=-1)[r1]

            big_tmp_y = torch.chunk(y,big_res,dim=-2)[r0]
            big_tmp_y = torch.chunk(big_tmp_y,big_res,dim=-1)[r1]

            res = max(round(big_tmp_x.size()[-2] / self.hparams['max_crop'][0] + 0.5), round(big_tmp_x.size()[-1] / self.hparams['max_crop'][1] + 0.5))
    
            outs = []
            for it0 in range(res):
                out0 = []
                for it1 in range(res):
                    tmp_x = torch.chunk(big_tmp_x,res,dim=-2)[it0]
                    tmp_x = torch.chunk(tmp_x,res,dim=-1)[it1]
                    #tmp_y = torch.chunk(y,res,dim=-2)[it0]
                    #tmp_y = torch.chunk(tmp_y,res,dim=-1)[it1]
        


                    #print('Memory:', get_gpu_memory_map()) - I chaked memory...

                    tmp_x = tmp_x.view(1, 3, tmp_x.size()[-2], tmp_x.size()[-1]).to(self.device)
                    #tmp_y = tmp_y.view(1, 3, tmp_y.size()[-2], tmp_y.size()[-1]).to(self.device)
                    tmp_x = self.generator(tmp_x)
                    out0.append(tmp_x.cpu())
                    #########################__GENERATOR__#############################
                    #self.generator.zero_grad()
    
                    #loss = self.criterion(tmp_x, tmp_y)
                    #loss.backward(retain_graph=True)
            

                    #self.opt.step()
                    ###################################################################
                    del tmp_x
                    #del tmp_y
            

                outs.append(torch.cat(out0, 3))
                del out0
            outs = torch.cat(outs, 2)#.clone()
            #####################################
            self.generator.zero_grad()

            loss = self.criterion(outs.cpu(),big_tmp_y)
            loss.backward()

            self.opt.step()
            #####################################
            del outs
            torch.cuda.empty_cache()
            #with torch.cuda.device('cuda'):
            #torch.cuda.empty_cache()
</code></pre>
<pre><code class="lang-auto">
My dataset (~same for both models):
"RandomPilTransforms" - is my class of same flips and deformations for sequential of images (so, here it useless, but still...)
I use save image with small quality on JPG and load it again, to simulate JPG defects, because I didn't find function, which do this without save. If you know the same functions, I'll be glad to hear about it.
</code></pre>
<p>class MyDataset(Dataset):<br>
def <strong>init</strong>(self, path, input_size, quality=50, max_len=False, save_path=r’data\cash’):</p>
<pre><code>	RPTargs = {
	'perspective': {
	'deformation': 0.5,
	'chance': 0.5,
	'resample': Image.BICUBIC,
	'fill': 0,
	'fillcolor': None
	},
	'flip_horizontal': 0.5,
	'flip_vertical': 0.5,
	'rotate_right': 0.5,
	'rotate_left': 0.5
	}
	self.transform = RandomPilTransforms(**RPTargs)

	
	self.path = path
	self.quality = quality
	self.save_path = save_path
	
	
	self.piltotensor = transforms.ToTensor()

	self.input_size = input_size

	self.names = []
	for dirpath,_,filenames in os.walk(path):
		for f in filenames:
			self.names.append(os.path.abspath(os.path.join(dirpath, f)))

	random.shuffle(self.names, random.seed())
	print('')
	print('All dataset:', len(self.names))


	
	
	self.len = len(self.names)
	if max_len:
		self.len = min(max_len, self.len)
	print('Using dataset:', self.len)
	print('')	
	

def __getitem__(self, index):
	try:
		y = Image.open(self.names[index]).convert('RGB')
	except:
		del self.names[index]
		return self.__getitem__(index)

	if y.size[0] * y.size[1] &gt; self.input_size[0] * self.input_size[1]:
		if y.size[0] &gt; y.size[1]:
			y = y.resize((self.input_size[0], self.input_size[0] * y.size[1] // y.size[0]))
		else:
			y = y.resize((self.input_size[1] * y.size[0] // y.size[1], self.input_size[1]))
			
	y = self.transform([y])[0]
	x = self.noisy(y)
	
	#y = cv2.convertScaleAbs(np.asarray(y))
	x = self.piltotensor(x)
	y = self.piltotensor(y)
	
	return x, y

def noisy(self, image):
	image.save(self.save_path  + r'\cash_img.jpg', quality=self.quality)
	image = Image.open(self.save_path  + r'\cash_img.jpg').convert('RGB')
	return image


def __len__(self):
	return self.len
</code></pre>
<pre><code class="lang-auto"></code></pre>