Transferring weights from plantnet pytorch model to keras model

marliesl: <p>I want to use the weights of a plantnet model, available online for pytorch, in keras. As I am only familiar with keras.<br>
The model architecture I am using is efficientnet b4. In keras, this model has a list of 611 weight tensors. The .tar file found online has of a list of 706 weight tensors. Removing the weight tensors with the name ‘…num_batches_tracked’ (I don’t think these are important) leaves me with 610 tensors. I changed the dimensions of the tensors and starting from the fourth tensors, there dimensions now match exactly. But the first three tensors of the keras_model (normalization_2/mean, normalization_2/variance, normalization_2/count) are missing. The pytorch tensors list also has two more tensors at the end. Can somebody explain if these weights are important or if I can initialize them randomly?<br>
Here is my code:</p>
<p>filename = “efficientnet_b4_weights_best_acc.tar”<br>
model = efficientnet_b4(weights=None)<br>
state_dict = torch.load(filename, map_location=torch.device(‘cpu’))[‘model’]<br>
model.load_state_dict(state_dict, strict=False)</p>
<p>keras_model = EfficientNetB4(<br>
include_top=False,<br>
weights=None,<br>
input_tensor=None,<br>
input_shape=None,<br>
pooling=None,<br>
classes=1081,<br>
classifier_activation=“softmax”<br>
)<br>
state_dict = {key: state_dict[key] for key in state_dict if not ‘num_batches_tracked’ in key}</p>
<p>keras_weights = [w.numpy() for w in state_dict.values()]<br>
for i in range(len(keras_weights)):<br>
if len(keras_weights[i].shape) == 4:<br>
keras_weights[i] = np.moveaxis(keras_weights[i], [0,1], [-1,-2])<br>
if keras_weights[i].shape[2] == 1:<br>
keras_weights[i] = np.moveaxis(keras_weights[i], 2, -1)<br>
if len(keras_weights[i].shape) == 2:<br>
keras_weights[i] = keras_weights[i].T</p>
<p>Thank you</p>

AbdulsalamBande: <p>Transferring PyTorch model weights to Keras can be challenging since the two frameworks use different formats to store their model weights. However, it is possible to transfer the weights using a few steps:</p>
<ol>
<li>Convert the PyTorch model to ONNX format: ONNX is an open format for representing deep learning models that can be used by various frameworks, including Keras. You can use the <code>torch.onnx.export()</code> function to convert your PyTorch model to ONNX format.</li>
<li>Convert the ONNX model to Keras format: You can use the <code>onnx2keras</code> package to convert the ONNX model to Keras format. This package provides a <code>onnx_to_keras()</code> function that can convert the ONNX model to a Keras model.</li>
<li>Load the weights into the Keras model: Once you have the Keras model, you can load the weights from the PyTorch model into the Keras model using the <code>model.set_weights()</code> function.</li>
</ol>
<p>Here is an example code snippet that shows how to transfer PyTorch model weights to Keras:</p>
<pre><code class="lang-auto">import torch
import onnx
from onnx2keras import onnx_to_keras

# Load PyTorch model
pytorch_model = torch.load('model.pt')

# Convert PyTorch model to ONNX format
dummy_input = torch.randn(1, 3, 224, 224)
torch.onnx.export(pytorch_model, dummy_input, 'model.onnx', input_names=['input'], output_names=['output'])

# Convert ONNX model to Keras format
onnx_model = onnx.load('model.onnx')
keras_model = onnx_to_keras(onnx_model, ['input'])

# Load weights into Keras model
weights = pytorch_model.state_dict()
keras_model.set_weights([weights[key].numpy() for key in weights.keys()])

# Use Keras model for inference
output = keras_model.predict(input_data)

</code></pre>

marliesl: <p>Hi<br>
Thank you very much for your response.</p>
<p>I ran into a value error during the convertion: ValueError: ‘/features/features.0/features.0.0/Conv_output_0_pad/’ is not a valid root scope name. A root scope name has to match the following pattern: ^[A-Za-z0-9.][A-Za-z0-9_.\/&gt;-]*$</p>
<p>I don’t know where this scope name is stored so I can change this…</p>

Bjorn_Lindqvist: <p>The problem is that PyTorch inserts slashes into names of layers and properties. And Keras does not like slashes in names of layers and properties. So if you want to use ONNX you have to somehow replace the slashes in the onnx file with some character Keras accepts.</p>