File size: 2,617 Bytes
e1bc4cb 4ec51d3 d55be37 db89d15 d55be37 41cb296 d55be37 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 |
---
license: cc-by-4.0
---
[PyTorch FasterRCNN](https://pytorch.org/vision/main/models/generated/torchvision.models.detection.fasterrcnn_resnet50_fpn.html#torchvision.models.detection.fasterrcnn_resnet50_fpn) with ResNet50 backbone finetuned on grayscale COCO.
The COCO images were transformed to grayscale using PIL. The hyperparameters and epochs were all kept the same as the implementation for [PyTorch](https://github.com/pytorch/vision/tree/main/references/detection#faster-r-cnn-resnet-50-fpn).
Can be used as pretrained model for multispectral imaging as suggested in this [paper](https://ceur-ws.org/Vol-2771/AICS2020_paper_50.pdf).
The file is given as a state_dict. Thus to initialize the model run:
```
# Load pretrained weights
state_dict = torch.load(model_path)['model']
# Load torchvision model
model = torchvision.models.detection.fasterrcnn_resnet50_fpn(weights=None)
# Adapt input convolution
model.backbone.body.conv1 = torch.nn.Conv2d(1, 64,
kernel_size=(7, 7), stride=(2, 2),
padding=(3, 3), bias=False).requires_grad_(True)
model.load_state_dict(state_dict)
```
If its going to be used for multispectral data, edit the first layer and duplicate the weights:
```
state_dict = torch.load(model_path, map_location=torch.device('cpu'))['model']
# Duplicate the weights
conv1_weight = state_dict['backbone.body.conv1.weight']
conv1_type = conv1_weight.dtype
conv1_weight = conv1_weight.float()
repeat = int(math.ceil(in_chans / 3))
conv1_weight = conv1_weight.repeat(1, repeat, 1, 1)[:, :in_chans, :, :]
#conv1_weight *= (3 / float(in_chans))
conv1_weight = conv1_weight.to(conv1_type)
state_dict['backbone.body.conv1.weight'] = conv1_weight
model.backbone.body.conv1 = torch.nn.Conv2d(in_chans, 64,
kernel_size=(7, 7), stride=(2, 2),
padding=(3, 3), bias=False).requires_grad_(True)
model.load_state_dict(state_dict)
```
For Faster-RCNN the input transform may need to be adapted. Here is an example:
```
coco_mean = [0.5] * in_chans
coco_std = [0.25] * in_chans
if in_chans > 3:
coco_mean[:3] = [0.485, 0.456, 0.406]
coco_std[:3] = [0.229, 0.224, 0.225]
transform = torchvision.models.detection.transform.GeneralizedRCNNTransform(min_size=800,
max_size=1333,
image_mean=coco_mean,
image_std=coco_std)
```
|