RuntimeError: Error(s) in loading state_dict for VGGModel:

Michael2: <p>Hello,</p>
<p>i have trained a NN-Model for Morph-Attack-Detection. When i load my trained Modell i getting the following error:</p>
<blockquote>
<p>Traceback (most recent call last):</p>
<p>File ~\anaconda3\Lib\site-packages\spyder_kernels\py3compat.py:356 in compat_exec<br>
exec(code, globals, locals)</p>
<p>File c:\users\micha\desktop\studium\8. semester\bachelor\algorithmen\gui\unbenannt1.py:227<br>
main()</p>
<p>File c:\users\micha\desktop\studium\8. semester\bachelor\algorithmen\gui\unbenannt1.py:216 in main<br>
process_folder()</p>
<p>File c:\users\micha\desktop\studium\8. semester\bachelor\algorithmen\gui\unbenannt1.py:106 in process_folder<br>
model.load_state_dict(torch.load(model_file_path, map_location=torch.device(‘cpu’)))</p>
<p>File ~\anaconda3\Lib\site-packages\torch\nn\modules\module.py:2153 in load_state_dict<br>
raise RuntimeError(‘Error(s) in loading state_dict for {}:\n\t{}’.format(</p>
<p>RuntimeError: Error(s) in loading state_dict for VGGModel:<br>
Missing key(s) in state_dict: “features.0.weight”, “features.0.bias”, “features.2.weight”, “features.2.bias”, “features.5.weight”, “features.5.bias”, “features.7.weight”, “features.7.bias”, “features.10.weight”, “features.10.bias”, “features.12.weight”, “features.12.bias”, “features.14.weight”, “features.14.bias”, “features.17.weight”, “features.17.bias”, “features.19.weight”, “features.19.bias”, “features.21.weight”, “features.21.bias”, “features.24.weight”, “features.24.bias”, “features.26.weight”, “features.26.bias”, “features.28.weight”, “features.28.bias”, “classifier.0.weight”, “classifier.0.bias”, “classifier.3.weight”, “classifier.3.bias”, “classifier.6.weight”, “classifier.6.bias”.</p>
</blockquote>
<p>My Main Code to load the model is:</p>
<p>import torch<br>
import torch.nn as nn<br>
import torchvision.models as models<br>
import torchvision.transforms as transforms<br>
from PIL import Image<br>
import pandas as pd<br>
import os<br>
from typing import Tuple<br>
import tkinter as tk<br>
from tkinter import filedialog, messagebox</p>
<h1><a name="funktion-zur-auswahl-einer-datei-1" class="anchor" href="#funktion-zur-auswahl-einer-datei-1"></a>Funktion zur Auswahl einer Datei</h1>
<p>def select_file(title, filetypes):<br>
root = tk.Tk()<br>
root.withdraw()<br>
file_path = filedialog.askopenfilename(title=title, filetypes=filetypes)<br>
return file_path</p>
<h1><a name="funktion-zur-auswahl-eines-ordners-2" class="anchor" href="#funktion-zur-auswahl-eines-ordners-2"></a>Funktion zur Auswahl eines Ordners</h1>
<p>def select_folder(title):<br>
root = tk.Tk()<br>
root.withdraw()<br>
folder_path = filedialog.askdirectory(title=title)<br>
return folder_path</p>
<h1><a name="definition-von-transform-3" class="anchor" href="#definition-von-transform-3"></a>Definition von transform</h1>
<p>transform = transforms.Compose([<br>
transforms.Resize((413, 531)),<br>
transforms.ToTensor(),<br>
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),<br>
])</p>
<h1><a name="verwendung-des-vortrainierten-vgg-modells-4" class="anchor" href="#verwendung-des-vortrainierten-vgg-modells-4"></a>Verwendung des vortrainierten VGG-Modells</h1>
<p>class VGGModel(nn.Module):<br>
def <strong>init</strong>(self, num_classes=1000):<br>
super(VGGModel, self).<strong>init</strong>()<br>
self.vgg = models.vgg16(pretrained=False)<br>
# Entfernen der letzten Schichten des vortrainierten Modells<br>
self.features = self.vgg.features<br>
self.avgpool = nn.AdaptiveAvgPool2d((7, 7))<br>
self.classifier = nn.Sequential(<br>
nn.Linear(512 * 7 * 7, 4096),<br>
nn.ReLU(True),<br>
nn.Dropout(),<br>
nn.Linear(4096, 4096),<br>
nn.ReLU(True),<br>
nn.Dropout(),<br>
nn.Linear(4096, num_classes)<br>
)</p>
<pre><code>def forward(self, x):
    x = self.features(x)
    x = self.avgpool(x)
    x = torch.flatten(x, 1)
    x = self.classifier(x)
    return x
</code></pre>
<h1><a name="funktion-zur-differential-morph-attack-detection-5" class="anchor" href="#funktion-zur-differential-morph-attack-detection-5"></a>Funktion zur Differential-Morph-Attack-Detection</h1>
<p>def detect_morph_differentially(suspected_morph_file_path: str, probe_face_file_path: str) → Tuple[str, bool, float]:<br>
return_code = “Success”<br>
is_morph = False<br>
score = 0.0</p>
<pre><code># Laden des verdächtigen Morph-Bildes
try:
    suspected_morph_image = Image.open(suspected_morph_file_path).convert('RGB')
    suspected_morph_image_tensor = transform(suspected_morph_image).unsqueeze(0)
except Exception as e:
    return f"Fehler beim Laden des verdächtigen Morph-Bildes: {e}", is_morph, score

# Laden des Original-Bildes
try:
    probe_face_image = Image.open(probe_face_file_path).convert('RGB')
    probe_face_image_tensor = transform(probe_face_image).unsqueeze(0)
except Exception as e:
    return f"Fehler beim Laden des Original-Bildes: {e}", is_morph, score

# Detection
with torch.no_grad():
    suspected_morph_output = model(suspected_morph_image_tensor)
    probe_face_output = model(probe_face_image_tensor)
    score = (suspected_morph_output + probe_face_output) / 2.0
    score = score.item()  # Convert tensor to float

# Setzen von is_morph=True, wenn das verdächtige Morph-Bild ein Morph ist, oder is_morph=False, wenn nicht
is_morph = score &gt; 0.5

return return_code, is_morph, score
</code></pre>
<h1><a name="hauptfunktion-zur-durchsuchung-des-ordners-und-durchfhrung-der-morph-detection-6" class="anchor" href="#hauptfunktion-zur-durchsuchung-des-ordners-und-durchfhrung-der-morph-detection-6"></a>Hauptfunktion zur Durchsuchung des Ordners und Durchführung der Morph-Detection</h1>
<p>def process_folder():<br>
model_file_path = select_file(“Wählen Sie die Modell-Datei aus”, [(“Modelldateien”, “*.pth <em>.pt")])<br>
morph_images_folder = select_folder(“Wählen Sie den Ordner mit den Morphbildern aus”)<br>
original_images_folder = select_folder(“Wählen Sie den Ordner mit den Originalbildern aus”)<br>
excel_file_path = select_file(“Wählen Sie die Excel-Liste aus”, [(“Excel-Dateien”, "</em>.xlsx”)])</p>
<pre><code># Laden des trainierten Modells
# Laden des trainierten Modells
model = VGGModel(num_classes=1000)  # Nummer der Klassen entsprechend anpassen
model.load_state_dict(torch.load(model_file_path, map_location=torch.device('cpu')))
model.eval()

# Laden der Excel-Liste
df = pd.read_excel(excel_file_path)

# Überprüfung und Anpassung der Excel-Liste
if 'Morph Detected 1' not in df.columns:
    df['Morph Detected 1'] = ''
if 'Score 1' not in df.columns:
    df['Score 1'] = 0.0
if 'Morph Detected 2' not in df.columns:
    df['Morph Detected 2'] = ''
if 'Score 2' not in df.columns:
    df['Score 2'] = 0.0

for idx, row in df.iterrows():
    try:
        morph_image_name = row['Dateiname']
        original_image_1_prefix = str(row['Bild1']).zfill(3)
        original_image_2_prefix = str(row['Bild2']).zfill(3)

        morph_image_path = os.path.join(morph_images_folder, morph_image_name)
        original_image_1_path = find_image_by_prefix(original_images_folder, original_image_1_prefix)
        original_image_2_path = find_image_by_prefix(original_images_folder, original_image_2_prefix)

        if original_image_1_path:
            # Morph-Detection 1
            _, is_morph_1, score_1 = detect_morph_differentially(morph_image_path, original_image_1_path)
            df.at[idx, 'Morph Detected 1'] = 'Wahr' if is_morph_1 else 'Falsch'
            df.at[idx, 'Score 1'] = score_1

        if original_image_2_path:
            # Morph-Detection 2
            _, is_morph_2, score_2 = detect_morph_differentially(morph_image_path, original_image_2_path)
            df.at[idx, 'Morph Detected 2'] = 'Wahr' if is_morph_2 else 'Falsch'
            df.at[idx, 'Score 2'] = score_2

    except Exception as e:
        print(f"Fehler bei der Verarbeitung der Zeile {idx}: {e}")

save_path = filedialog.asksaveasfilename(defaultextension=".xlsx", filetypes=[("Excel-Dateien", "*.xlsx")])
if save_path:
    df.to_excel(save_path, index=False)
    messagebox.showinfo("Erfolg", "Die Excel-Liste wurde erfolgreich gespeichert.")
else:
    messagebox.showwarning("Abgebrochen", "Speichern der Excel-Liste abgebrochen.")
</code></pre>
<h1><a name="funktion-zum-finden-eines-bildes-anhand-des-prfixes-7" class="anchor" href="#funktion-zum-finden-eines-bildes-anhand-des-prfixes-7"></a>Funktion zum Finden eines Bildes anhand des Präfixes</h1>
<p>def find_image_by_prefix(folder_path, prefix):<br>
for file_name in os.listdir(folder_path):<br>
if file_name.startswith(prefix):<br>
return os.path.join(folder_path, file_name)<br>
return None</p>
<h1><a name="einzelbilduntersuchung-8" class="anchor" href="#einzelbilduntersuchung-8"></a>Einzelbilduntersuchung</h1>
<p>def single_image_investigation():<br>
model_file_path = select_file(“Wählen Sie die Modell-Datei aus”, [(“Modelldateien”, “*.pth <em>.pt")])<br>
morph_image_path = select_file(“Wählen Sie das Morphbild aus”, [(“Bilddateien”, "</em>.jpg *.jpeg <em>.png")])<br>
probe_face_image_path = select_file(“Wählen Sie das Originalbild aus”, [(“Bilddateien”, "</em>.jpg *.jpeg *.png”)])</p>
<pre><code># Laden des trainierten Modells
global model
model = VGGModel()
model.load_state_dict(torch.load(model_file_path, map_location=torch.device('cpu')))
model.eval()

# Morph-Detection
return_code, is_morph, score = detect_morph_differentially(morph_image_path, probe_face_image_path)

# Anzeige der Ergebnisse
if return_code == "Success":
    result_message = f"Is Morph: {is_morph}, Score: {score}"
else:
    result_message = return_code

# Laden und Anzeigen der Bilder
morph_image = Image.open(morph_image_path)
probe_face_image = Image.open(probe_face_image_path)

# Erstellen des Fensters
root = tk.Tk()
root.title("Morph Detection Result")

# Anzeigen des Ergebnisses
tk.Label(root, text=result_message, font=("Helvetica", 16)).pack()

# Funktion zur Anzeige der Bilder
def display_images():
    morph_image_tk = ImageTk.PhotoImage(morph_image)
    probe_face_image_tk = ImageTk.PhotoImage(probe_face_image)
    morph_label = tk.Label(root, image=morph_image_tk)
    probe_label = tk.Label(root, image=probe_face_image_tk)
    morph_label.image = morph_image_tk  # Referenz behalten, um das Bild im Fenster anzuzeigen
    probe_label.image = probe_face_image_tk
    morph_label.pack()
    probe_label.pack()

display_images()
root.mainloop()
</code></pre>
<h1><a name="hauptfunktion-zum-ausfhren-des-skripts-9" class="anchor" href="#hauptfunktion-zum-ausfhren-des-skripts-9"></a>Hauptfunktion zum Ausführen des Skripts</h1>
<p>def main():<br>
root = tk.Tk()<br>
root.withdraw()</p>
<pre><code># Dialog für die Auswahl zwischen Ordner oder Einzelbilduntersuchung
choice = messagebox.askquestion("Auswahl", "Möchten Sie einen Ordner durchsuchen oder eine Einzelbilduntersuchung durchführen?")

if choice == 'yes':
    process_folder()
else:
    single_image_investigation()
</code></pre>
<p>if <strong>name</strong> == “<strong>main</strong>”:<br>
main()</p>
<p>And this is my Code to train the Model:</p>
<blockquote>
<p>import torch<br>
import torch.nn as nn<br>
import torch.optim as optim<br>
import torchvision<br>
import torchvision.transforms as transforms<br>
import torchvision.models as models<br>
from torch.utils.data import DataLoader<br>
from PIL import Image, ImageTk<br>
import tkinter as tk<br>
from tkinter import filedialog, messagebox<br>
import os</p>
<h1>Parameter</h1>
<p>batch_size = 32<br>
learning_rate = 0.001<br>
num_epochs = 10<br>
num_classes = 1000  # Anzahl der Klassen im Datensatz</p>
<h1>Datenverarbeitung</h1>
<p>transform = transforms.Compose([<br>
transforms.Resize((413, 531)),<br>
transforms.ToTensor(),<br>
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),<br>
])</p>
<h1>Funktion zur Auswahl einer Datei</h1>
<p>def select_file(title, filetypes):<br>
root = tk.Tk()<br>
root.withdraw()<br>
file_path = filedialog.askopenfilename(title=title, filetypes=filetypes)<br>
return file_path</p>
<h1>Funktion zur Auswahl eines Ordners</h1>
<p>def select_folder(title):<br>
root = tk.Tk()<br>
root.withdraw()<br>
folder_path = filedialog.askdirectory(title=title)<br>
return folder_path</p>
<h1>VGG-Modell definieren</h1>
<p>class VGGModel(nn.Module):<br>
def <strong>init</strong>(self, num_classes):<br>
super(VGGModel, self).<strong>init</strong>()<br>
self.vgg = models.vgg16(pretrained=True)<br>
self.vgg.classifier[6] = nn.Linear(4096, num_classes)  # Anpassung der letzten FC-Schicht</p>
<pre><code>def forward(self, x):
    x = self.vgg(x)
    return x
</code></pre>
<p>model = VGGModel(num_classes=num_classes)</p>
<h1>Loss und Optimizer</h1>
<p>criterion = nn.CrossEntropyLoss()<br>
optimizer = optim.Adam(model.parameters(), lr=learning_rate)</p>
<h1>Training des Modells</h1>
<p>def train_model(model, train_loader, val_loader, criterion, optimizer, num_epochs):<br>
for epoch in range(num_epochs):<br>
model.train()<br>
running_loss = 0.0<br>
correct = 0<br>
total = 0</p>
<pre><code>    for inputs, labels in train_loader:
        optimizer.zero_grad()
        outputs = model(inputs)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()

        running_loss += loss.item() * inputs.size(0)
        _, predicted = torch.max(outputs, 1)
        total += labels.size(0)
        correct += (predicted == labels).sum().item()

    epoch_loss = running_loss / len(train_loader.dataset)
    epoch_acc = correct / total
    print(f'Epoch {epoch}/{num_epochs-1}, Loss: {epoch_loss:.4f}, Accuracy: {epoch_acc:.4f}')

    # Validation
    model.eval()
    val_loss = 0.0
    val_correct = 0
    val_total = 0
    with torch.no_grad():
        for inputs, labels in val_loader:
            outputs = model(inputs)
            loss = criterion(outputs, labels)
            val_loss += loss.item() * inputs.size(0)
            _, predicted = torch.max(outputs, 1)
            val_total += labels.size(0)
            val_correct += (predicted == labels).sum().item()

    val_epoch_loss = val_loss / len(val_loader.dataset)
    val_epoch_acc = val_correct / val_total
    print(f'Validation Loss: {val_epoch_loss:.4f}, Validation Accuracy: {val_epoch_acc:.4f}')
</code></pre>
<h1>Hauptfunktion zur Auswahl der Pfade und zum Training des Modells</h1>
<p>def main():<br>
train_data_path = select_folder(“Wählen Sie den Ordner mit den Trainingsdaten aus”)<br>
val_data_path = select_folder(“Wählen Sie den Ordner mit den Validierungsdaten aus”)<br>
save_model_path = filedialog.asksaveasfilename(defaultextension=“.pth”, filetypes=[(“PyTorch Model”, “*.pth”)], title=“Wählen Sie den Speicherort für das trainierte Modell aus”)</p>
<pre><code>if not train_data_path or not val_data_path or not save_model_path:
    messagebox.showwarning("Abgebrochen", "Einer der Pfade wurde nicht ausgewählt. Das Training wurde abgebrochen.")
    return

# Daten laden
train_data = torchvision.datasets.ImageFolder(root=train_data_path, transform=transform)
train_loader = DataLoader(train_data, batch_size=batch_size, shuffle=True)

val_data = torchvision.datasets.ImageFolder(root=val_data_path, transform=transform)
val_loader = DataLoader(val_data, batch_size=batch_size, shuffle=False)

# Modell trainieren
train_model(model, train_loader, val_loader, criterion, optimizer, num_epochs)

# Modell speichern
torch.save(model.state_dict(), save_model_path)
messagebox.showinfo("Erfolg", f"Das Modell wurde erfolgreich gespeichert unter: {save_model_path}")
</code></pre>
<p>if <strong>name</strong> == “<strong>main</strong>”:<br>
main()</p>
</blockquote>
<p>I would be very grateful for some help, as this problem has been bothering me for a long time</p>

ptrblck: <p>Your model definitions are not equal.<br>
In the first part of the code you are creating a <code>self.features</code> attribute:</p>
<pre data-code-wrap="python"><code class="lang-python">class VGGModel(nn.Module):
    def __init__(self, num_classes=1000):
        super(VGGModel, self).__init__()
        self.vgg = models.vgg16(pretrained=False)
        # Entfernen der letzten Schichten des vortrainierten Modells
        self.features = self.vgg.features
        self.avgpool = nn.AdaptiveAvgPool2d((7, 7))
        self.classifier = nn.Sequential(
        nn.Linear(512 * 7 * 7, 4096),
        nn.ReLU(True),
        nn.Dropout(),
        nn.Linear(4096, 4096),
        nn.ReLU(True),
        nn.Dropout(),
        nn.Linear(4096, num_classes)
        )
...
</code></pre>
<p>while it’s missing in the second model definition:</p>
<pre data-code-wrap="python"><code class="lang-python">class VGGModel(nn.Module):
    def __init__(self, num_classes):
        super(VGGModel, self).__init__()
        self.vgg = models.vgg16(pretrained=True)
        self.vgg.classifier[6] = nn.Linear(4096, num_classes) # Anpassung der letzten FC-Schicht
...
</code></pre>
<p>Reuse the same model definition and it should work.</p>

Michael2: <p>Thank you ptrblck for your answer. It was helped mit to fix the problem . Thank you very much. But now now I have the problem that when I want to compare two images for their similarity, the model gives a different score each time, although I think it should be the same each time it is checked. I think it is due to the following warnings: Warning: conv1.weight not found in provided state_dict.<br>
Warning: bn1.weight not found in provided state_dict.<br>
Warning: bn1.bias not found in provided state_dict.<br>
Warning: bn1.running_mean not found in provided state_dict.<br>
Warning: bn1.running_var not found in provided state_dict.<br>
Warning: bn1.num_batches_tracked not found in provided state_dict.<br>
Warning: layer1.0.conv1.weight not found in provided state_dict.<br>
Warning: layer1.0.bn1.weight not found in provided state_dict.<br>
Warning: layer1.0.bn1.bias not found in provided state_dict.<br>
Warning: layer1.0.bn1.running_mean not found in provided state_dict.<br>
Warning: layer1.0.bn1.running_var not found in provided state_dict.<br>
Warning: layer1.0.bn1.num_batches_tracked not found in provided state_dict.</p>
<p>This ist my Code to load the model:</p>
<h1><a name="definition-von-transform-1" class="anchor" href="#definition-von-transform-1"></a>Definition von transform</h1>
<p>transform = transforms.Compose([<br>
transforms.Resize((128, 128)),<br>
transforms.ToTensor(),<br>
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),<br>
])</p>
<h1><a name="definition-von-morphdetectionresnet-als-klasse-2" class="anchor" href="#definition-von-morphdetectionresnet-als-klasse-2"></a>Definition von MorphDetectionResNet als Klasse</h1>
<p>class MorphDetectionResNet(nn.Module):<br>
def <strong>init</strong>(self):<br>
super(MorphDetectionResNet, self).<strong>init</strong>()<br>
self.model = models.resnet50(weights=models.ResNet50_Weights.IMAGENET1K_V1)<br>
self.model.fc = nn.Linear(self.model.fc.in_features, 2)  # assuming binary classification: Morph (0) or Bona-Fida (1)<br>
self.softmax = nn.Softmax(dim=1)</p>
<pre><code>def forward(self, x):
    x = self.model(x)
    x = self.softmax(x)
    return x
</code></pre>
<h1><a name="funktion-zum-laden-des-modell-zustands-mit-bereinstimmung-3" class="anchor" href="#funktion-zum-laden-des-modell-zustands-mit-bereinstimmung-3"></a>Funktion zum Laden des Modell-Zustands mit Übereinstimmung</h1>
<p>def load_state_dict_with_match(model, state_dict):<br>
model_state_dict = model.state_dict()<br>
new_state_dict = OrderedDict()</p>
<pre><code>for key, value in state_dict.items():
    if key in model_state_dict:
        new_state_dict[key] = value
    else:
        print(f"Warning: {key} not found in provided state_dict.")

# Load the state_dict into the model
model.load_state_dict(new_state_dict)

# Lade den gefilterten state_dict
model.load_state_dict(new_state_dict, strict=False)
</code></pre>
<p>Can you help me by this problem. Thank you very much</p>

ptrblck: <aside class="quote no-group" data-username="Michael2" data-post="3" data-topic="206014">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/michael2/48/61717_2.png" class="avatar"> Michael2:</div>
<blockquote>
<p>I think it is due to the following warnings: Warning: conv1.weight not found in provided state_dict.</p>
</blockquote>
</aside>
<p>Yes, the explanation sounds reasonable. If parameters are ignored as they are missing in the provided <code>state_dict</code>, I would expect to see different results.</p>
<p>I don’t know where your current <code>state_dict</code> comes from, but apparently it does not contain the same keys.<br>
Your code snippet only shows how the <code>MorphDetectionResNet</code> is defined and how you are trying to load a <code>state_dict</code>.</p>