Different results everytime calling forward()

lcdreamer0621: <p>import os<br>
import torch<br>
from torch import nn<br>
from torch.utils.data import DataLoader, Dataset<br>
import numpy as np<br>
import pandas as pd<br>
import random<br>
import matplotlib.colors as mcolors<br>
import matplotlib.pyplot as plt<br>
from sklearn.neighbors import NearestNeighbors</p>
<h1><a name="set-random-seeds-for-reproducibility-1" class="anchor" href="#set-random-seeds-for-reproducibility-1"></a>Set random seeds for reproducibility</h1>
<p>seed = 42<br>
torch.manual_seed(seed)<br>
torch.cuda.manual_seed(seed)<br>
torch.cuda.manual_seed_all(seed)<br>
np.random.seed(seed)<br>
random.seed(seed)</p>
<h1><a name="ensure-deterministic-behavior-2" class="anchor" href="#ensure-deterministic-behavior-2"></a>Ensure deterministic behavior</h1>
<p>torch.backends.cudnn.deterministic = True<br>
torch.backends.cudnn.benchmark = False</p>
<p>class EMGDataset(Dataset):<br>
def <strong>init</strong>(self, emg_data):<br>
emg_data = emg_data.astype(np.float32)<br>
self.x = np.array(emg_data)<br>
self.n_samples = self.x.shape[0]</p>
<pre><code>def __getitem__(self, index):
    return self.x[index]

def __len__(self):
    return self.n_samples
</code></pre>
<p>def create_dataloaders():<br>
concat_all_mus = pd.read_csv(“EMG_dataset_prep.csv”, index_col=0)<br>
dataset = EMGDataset(concat_all_mus.iloc[:, :1200])<br>
dataloader = DataLoader(dataset, batch_size=len(dataset), shuffle=False, num_workers=0)<br>
return dataloader</p>
<h1><a name="ensure-dataloader-consistency-3" class="anchor" href="#ensure-dataloader-consistency-3"></a>Ensure DataLoader consistency</h1>
<p>whole_data = create_dataloaders()</p>
<h1><a name="define-device-based-on-availability-4" class="anchor" href="#define-device-based-on-availability-4"></a>Define device based on availability</h1>
<p>device = ‘cuda’ if torch.cuda.is_available() else ‘cpu’</p>
<h1><a name="load-model-and-set-to-evaluation-mode-5" class="anchor" href="#load-model-and-set-to-evaluation-mode-5"></a>Load model and set to evaluation mode</h1>
<p>model = model_builder.VAE(input_shape=1200, hidden_shape=512, output_shape=2, device=device)<br>
model.load_state_dict(torch.load(‘models/all_dataVAE_0.001_5000_512_2_beta_0.1.pth’, map_location=device))<br>
model.to(device)<br>
model.eval()</p>
<h1><a name="define-colors-6" class="anchor" href="#define-colors-6"></a>Define colors</h1>
<p>colors = list(mcolors.CSS4_COLORS.keys())<br>
selected_colors = colors[:130]</p>
<p>def plot_latent_space(example_index, whole_data, model, not_all):<br>
data_iter = iter(whole_data)<br>
data_sample = next(data_iter)<br>
data_sample = data_sample.to(device)</p>
<pre><code>model.eval()
with torch.no_grad():
    z, latent_mean, latent_std, reconstructed, _ = model(data_sample)

result = z_index(z.cpu(), index_dataframe)
choose_index = result[result['subject'].isin(example_index)]

print(f"Latent space mean: {np.mean(z.cpu().detach().numpy(), axis=0)}")
print(f"Latent space std: {np.std(z.cpu().detach().numpy(), axis=0)}")

plt.figure(figsize=(10, 10))
for idx in choose_index['subject'].unique():
    subset = choose_index[choose_index['subject'] == idx]
    plt.scatter(subset['z1'], subset['z2'], label=f'Index {idx}', c=selected_colors[idx])

if not_all:
    plt.legend(title='Index')

plt.xlabel('VAE_axis_1')
plt.ylabel('VAE_axis_2')
plt.title('Latent Space Result')
plt.show()
</code></pre>
<p>def z_index(z, index_dataframe):<br>
result = pd.concat([pd.DataFrame(z.numpy(), columns=[‘z1’, ‘z2’]), index_dataframe], axis=1)<br>
return result</p>
<h1><a name="load-index-data-7" class="anchor" href="#load-index-data-7"></a>Load index data</h1>
<p>concat_all_mus = pd.read_csv(“EMG_dataset_index.csv”, index_col=0)<br>
index_dataframe = concat_all_mus.iloc[:, 1200:1201]<br>
all_index = np.unique(index_dataframe).tolist()</p>
<h1><a name="call-plot_latent_space-function-twice-to-check-for-consistency-8" class="anchor" href="#call-plot_latent_space-function-twice-to-check-for-consistency-8"></a>Call plot_latent_space function twice to check for consistency</h1>
<p>plot_latent_space(all_index, whole_data, model, 0)<br>
plot_latent_space(all_index, whole_data, model, 0)</p>
<p>I have a question about this code. When I call function plot_latent_space(), It gives different results. I was expecting it would give a equal result but it does not. Why does that happen?</p>
<p>Latent space mean: [0.04162313 0.03197911]<br>
Latent space std: [0.9312542 0.911205 ]</p>
<p>Latent space mean: [0.03519088 0.02695682]<br>
Latent space std: [0.938787   0.91289806]</p>

soulitzer: <p>What if you set random seed inside the plot_latent_space function?</p>

lcdreamer0621: <p>Thanks! It worked. Do you know the reason why the seed should be set inside the function?</p>

soulitzer: <p>If you are using random operations in the function it will advance the random state so that the next time you execute the function the random operations would produce different results.</p>