Dataset creation for noisy data

username: arshishir
cooked: <p>Hi,</p>
<p>I am trying to create a noisy dataset for ML. Here’s what I did:</p>
<pre><code class="lang-auto">mnist_train = MNIST('../data/MNIST', download = True,
                    transform = transforms.Compose([
                        transforms.ToTensor(),
                    ]), train = True)

mnist_test = MNIST('../data/MNIST', download = True,
                    transform = transforms.Compose([
                        transforms.ToTensor(),
                    ]), train = False)
</code></pre>
<p>And this:</p>
<pre><code class="lang-auto">def add_noise_sp(image,sd,amount=0.2):
    np.random.seed(seed=sd)

    low_clip = 0.13
    std = 0.31

    image = np.asarray(image)
    out = image.copy()
    p = amount
    q = 0.5 
    flipped = np.random.choice([True, False], size=image.shape,
                                   p=[p, 1 - p])
    salted = np.random.choice([True, False], size=image.shape,
                                  p=[q, 1 - q])
    peppered = ~salted
    out[flipped &amp; salted] = low_clip-2*std
    
    return torch.tensor(out)

s = 1
class SyntheticNoiseDatasetsalttr(Dataset):
    def __init__(self, data, mode='train'):
        self.mode = mode
        self.data = data
        
    def __len__(self):
        return len(self.data)
    
    def __getitem__(self, index):
        global s
        img = self.data[index][0]
        s = s + 1 
        return add_noise_sp(img,s), img
    
b = 3425
class SyntheticNoiseDatasetsaltte(Dataset):
    def __init__(self, data, mode='test'):
        self.mode = mode
        self.data = data
        
    def __len__(self):
        return len(self.data)
    
    def __getitem__(self, index):
        global b
        img = self.data[index][0]
        b = b + 1 
        return add_noise_sp(img,b), img
</code></pre>
<p>This how I create the dataset:</p>
<pre><code class="lang-auto">noisy_mnist_train = SyntheticNoiseDatasetsalttr(mnist_train, 'train')
noisy_mnist_test = SyntheticNoiseDatasetsaltte(mnist_test, 'test')
train_set, val_set = torch.utils.data.random_split(noisy_mnist_train, [55000, 5000], generator=torch.Generator().manual_seed(42))
</code></pre>
<p>I thought this would give me a constant dataset. Does this mean that when I train my neural net, it will see different noisy version of the same input everytime? When I run the following command, I get digit 5 but everytime it has different noise on it. It seems like everytime I call DataLoader, it’s running my datageneration code and giving me different noisy data of the same subject.</p>
<pre><code class="lang-auto">torch.manual_seed(123)
bt = DataLoader(val_set,batch_size=1, shuffle= True)
bt =next(iter(bt))
noisy,clean = bt
plt.imshow(noisy.squeeze(), cmap='gray')
</code></pre>

username: ptrblck
cooked: <p>The posted code doesn’t show the repeated calls, but I assume you are just executing the 5 lines of code in a REPL multiple times. If so, then the different noise levels would be expected, since you are using global variables for the seeds (<code>s</code> and <code>b</code>), which are updated in each call to <code>__getitem__</code>.</p>

username: arshishir
cooked: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a>,</p>
<p>Thanks for the reply. I mean if I execute the last block multiple of times, I can get a different output. I thought by running the code below my dataset will be generated.</p>
<pre><code class="lang-auto">noisy_mnist_train = SyntheticNoiseDatasetsalttr(mnist_train, 'train')
noisy_mnist_test = SyntheticNoiseDatasetsaltte(mnist_test, 'test')
train_set, val_set = torch.utils.data.random_split(noisy_mnist_train, [55000, 5000], generator=torch.Generator().manual_seed(42))
</code></pre>
<p>Then that dataset is fixed. When I run the block below any number of times just by hitting shift+enter in jupyter notebook I should get the same output.</p>
<pre><code class="lang-auto">torch.manual_seed(123)
bt = DataLoader(val_set,batch_size=1, shuffle= True)
bt =next(iter(bt))
noisy,clean = bt
plt.imshow(noisy.squeeze(), cmap='gray')
</code></pre>
<p>All I want  is to generate noisy data with the seed value <code>s</code> and <code>b</code>.</p>

username: InnovArul
cooked: <aside class="quote no-group" data-username="arshishir" data-post="1" data-topic="126249">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/arshishir/48/40111_2.png" class="avatar"> arshishir:</div>
<blockquote>
<pre><code class="lang-auto">    def __getitem__(self, index):
        global b
        img = self.data[index][0]
        b = b + 1 
        return add_noise_sp(img,b), img
</code></pre>
</blockquote>
</aside>
<p>You are adding noise while getting every new item in these lines.  If you want a fixed noise, one way is to add noise to tensors manually and use <code>TensorDataset</code>.</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="arshishir" data-post="3" data-topic="126249">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/arshishir/48/40111_2.png" class="avatar"> arshishir:</div>
<blockquote>
<p>I thought by running the code below my dataset will be generated.</p>
</blockquote>
</aside>
<p>Yes, your datasets will be recreated, but note that you are explicitly using <code>global</code> seeds, which will not be recreated.<br>
As mentioned before, either reset the seed inside the datasets or use <a class="mention" href="/u/innovarul">@InnovArul</a> approach, in case you want to add a fixed noise to each sample.</p>

username: arshishir
cooked: <p>For each picture, I want different noise. However, every time my dataset is created I want the same noise for the picture (hence I am trying to set seed for each picture). So the problem is the <code>global</code> seed, right? This global seed does not get reset, right? Thanks for the suggestions, I have never used <code>TensorDataset</code> but I will look into <code>TensorDataset</code></p>

username: arshishir
cooked: <p>How can I reset the <code>seed</code> inside the datasets?</p>

username: ptrblck
cooked: <p>You could e.g. set the initial seed in the <code>__init__</code> method and update it in the <code>__getitem__</code>.<br>
This would make sure to initialize each new dataset with the same seed.</p>

username: arshishir
cooked: <p>I did this:</p>
<pre><code class="lang-auto">class SyntheticNoiseDatasetsalttr(Dataset):
    def __init__(self, data,mode='train'):
        self.mode = mode
        self.data = data
        self.s = 1
        
    def __len__(self):
        return len(self.data)
    
    def __getitem__(self, index):
        img = self.data[index][0]
        self.s = self.s + 1 
        return add_noise_sp(img,self.s), img
</code></pre>
<p>It seems like I am making a mistake here because I get the same result as before: every time I run the code I get different noise:<br>
1st run:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/b/3/b3e026b2bfce3f1ee98c4b95fae22201a381428e.png" alt="image" data-base62-sha1="pFfHWSK6RsEAi8hgHhYHOJbA3r8" width="251" height="248"><br>
2nd run:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/1/8/182b27f2ea5a248bbdc9f11bd3743be6bbf2f9d8.png" alt="image" data-base62-sha1="3rNV5b7hyJWtEeVDSyhGfRGsajS" width="251" height="248"></p>
<p>All I want is different noise in different images (controlled by the seed) but it should be the same every time I execute the code.</p>

username: ptrblck
cooked: <p>I cannot reproduce the issue using:</p>
<pre><code class="lang-python">def add_noise_sp(image,sd,amount=0.2):
    np.random.seed(seed=sd)

    low_clip = 0.13
    std = 0.31

    image = np.asarray(image)
    out = image.copy()
    p = amount
    q = 0.5 
    flipped = np.random.choice([True, False], size=image.shape,
                                   p=[p, 1 - p])
    salted = np.random.choice([True, False], size=image.shape,
                                  p=[q, 1 - q])
    peppered = ~salted
    out[flipped &amp; salted] = low_clip-2*std
    
    return torch.tensor(out)

class SyntheticNoiseDatasetsalttr(torch.utils.data.Dataset):
    def __init__(self, data,mode='train'):
        self.mode = mode
        self.data = data
        self.s = 1
        
    def __len__(self):
        return len(self.data)
    
    def __getitem__(self, index):
        img = self.data[index][0]
        self.s = self.s + 1 
        return add_noise_sp(img,self.s), img
    
mnist_train = torchvision.datasets.MNIST(
    root='./data', transform=torchvision.transforms.ToTensor())
noisy_mnist_train = SyntheticNoiseDatasetsalttr(mnist_train, 'train')
train_set, val_set = torch.utils.data.random_split(noisy_mnist_train, [55000, 5000], generator=torch.Generator().manual_seed(42))

torch.manual_seed(123)
bt = torch.utils.data.DataLoader(val_set,batch_size=1, shuffle= True)
bt =next(iter(bt))
noisy,clean = bt
plt.imshow(noisy.squeeze(), cmap='gray')
</code></pre>
<p>and get always the same output by rerunning the entire code.<br>
I don’t know which part you are rerunning, but make sure to reinitialize the dataset.</p>