DETR+RoBERTaを使ったマルチモーダル推論モデル「MDETR」を使ってみる (推論できず)

# タイトル: DETR+RoBERTaを使ったマルチモーダル推論モデル「MDETR」を使ってみる

DETR+RoBERTaを使ったマルチモーダル推論モデル「MDETR」が公開されているので、Google Colab上で使ってみました。

しかし、手順通り、MDETRの「Visual Question Answering(視覚的質問応答)の推論」の部分を実行しても、結果出力データにnan値が多く含まれ、推論結果もnan値となり、推論が出来ませんでした。以降に試行結果を記載します。

# 下記の記事の通りにMDETRを実行

画像について何でも文章で質問して、その回答(該当する単語)が得られるという、MDETRの機能が面白そうなので、下記の記事にある、MDETRの「Visual Question Answering(視覚的質問応答)の推論」の部分を実行してみます。

複数モダリティを使う「マルチモーダルAI」の紹介
第1回 DETR+RoBERTaを使ったマルチモーダル推論モデル「MDETR」の紹介とColabでの推論
https://www.ogis-ri.co.jp/otc/hiroba/technical/multimodal-ai/part1.html

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/606924/e7dd524b-38a8-47e0-a2d8-4c50b3c50a08.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/606924/37c7eda5-ad49-46b2-9227-ed91a6f0f637.png)

上記記事の元になっているチュートリアルファイル
https://colab.research.google.com/github/ashkamath/mdetr/blob/colab/notebooks/MDETR_demo.ipynb

## Google Colabを用意・GPU利用へ設定変更

新規にGoogle Colabを開いて、GPU利用へ設定を変更
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/606924/cf705af2-f8c7-4ceb-807b-18354e58b6d9.png)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/606924/2f85ba6b-d738-485d-a596-c6a5eeb216e9.png)

## (記事通り) 4-1.必要なパッケージの準備と前処理･後処理の定義

記事の通りに実行してみます。(コード・実行結果のみ抜粋)


```python
!pip install timm transformers
```

    Requirement already satisfied: timm in /usr/local/lib/python3.11/dist-packages (1.0.15)
    Requirement already satisfied: transformers in /usr/local/lib/python3.11/dist-packages (4.50.3)
    Requirement already satisfied: torch in /usr/local/lib/python3.11/dist-packages (from timm) (2.6.0+cu124)
    Requirement already satisfied: torchvision in /usr/local/lib/python3.11/dist-packages (from timm) (0.21.0+cu124)
    Requirement already satisfied: pyyaml in /usr/local/lib/python3.11/dist-packages (from timm) (6.0.2)
    Requirement already satisfied: huggingface_hub in /usr/local/lib/python3.11/dist-packages (from timm) (0.30.1)
    Requirement already satisfied: safetensors in /usr/local/lib/python3.11/dist-packages (from timm) (0.5.3)
    Requirement already satisfied: filelock in /usr/local/lib/python3.11/dist-packages (from transformers) (3.18.0)
    Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.11/dist-packages (from transformers) (2.0.2)
    Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.11/dist-packages (from transformers) (24.2)
    Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.11/dist-packages (from transformers) (2024.11.6)
    Requirement already satisfied: requests in /usr/local/lib/python3.11/dist-packages (from transformers) (2.32.3)
    Requirement already satisfied: tokenizers<0.22,>=0.21 in /usr/local/lib/python3.11/dist-packages (from transformers) (0.21.1)
    Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.11/dist-packages (from transformers) (4.67.1)
    Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.11/dist-packages (from huggingface_hub->timm) (2025.3.2)
    Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.11/dist-packages (from huggingface_hub->timm) (4.13.1)
    Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.11/dist-packages (from requests->transformers) (3.4.1)
    Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.11/dist-packages (from requests->transformers) (3.10)
    Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.11/dist-packages (from requests->transformers) (2.3.0)
    Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.11/dist-packages (from requests->transformers) (2025.1.31)
    Requirement already satisfied: networkx in /usr/local/lib/python3.11/dist-packages (from torch->timm) (3.4.2)
    Requirement already satisfied: jinja2 in /usr/local/lib/python3.11/dist-packages (from torch->timm) (3.1.6)
    Collecting nvidia-cuda-nvrtc-cu12==12.4.127 (from torch->timm)
      Downloading nvidia_cuda_nvrtc_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
    Collecting nvidia-cuda-runtime-cu12==12.4.127 (from torch->timm)
      Downloading nvidia_cuda_runtime_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
    Collecting nvidia-cuda-cupti-cu12==12.4.127 (from torch->timm)
      Downloading nvidia_cuda_cupti_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl.metadata (1.6 kB)
    Collecting nvidia-cudnn-cu12==9.1.0.70 (from torch->timm)
      Downloading nvidia_cudnn_cu12-9.1.0.70-py3-none-manylinux2014_x86_64.whl.metadata (1.6 kB)
    Collecting nvidia-cublas-cu12==12.4.5.8 (from torch->timm)
      Downloading nvidia_cublas_cu12-12.4.5.8-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
    Collecting nvidia-cufft-cu12==11.2.1.3 (from torch->timm)
      Downloading nvidia_cufft_cu12-11.2.1.3-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
    Collecting nvidia-curand-cu12==10.3.5.147 (from torch->timm)
      Downloading nvidia_curand_cu12-10.3.5.147-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
    Collecting nvidia-cusolver-cu12==11.6.1.9 (from torch->timm)
      Downloading nvidia_cusolver_cu12-11.6.1.9-py3-none-manylinux2014_x86_64.whl.metadata (1.6 kB)
    Collecting nvidia-cusparse-cu12==12.3.1.170 (from torch->timm)
      Downloading nvidia_cusparse_cu12-12.3.1.170-py3-none-manylinux2014_x86_64.whl.metadata (1.6 kB)
    Requirement already satisfied: nvidia-cusparselt-cu12==0.6.2 in /usr/local/lib/python3.11/dist-packages (from torch->timm) (0.6.2)
    Requirement already satisfied: nvidia-nccl-cu12==2.21.5 in /usr/local/lib/python3.11/dist-packages (from torch->timm) (2.21.5)
    Requirement already satisfied: nvidia-nvtx-cu12==12.4.127 in /usr/local/lib/python3.11/dist-packages (from torch->timm) (12.4.127)
    Collecting nvidia-nvjitlink-cu12==12.4.127 (from torch->timm)
      Downloading nvidia_nvjitlink_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
    Requirement already satisfied: triton==3.2.0 in /usr/local/lib/python3.11/dist-packages (from torch->timm) (3.2.0)
    Requirement already satisfied: sympy==1.13.1 in /usr/local/lib/python3.11/dist-packages (from torch->timm) (1.13.1)
    Requirement already satisfied: mpmath<1.4,>=1.1.0 in /usr/local/lib/python3.11/dist-packages (from sympy==1.13.1->torch->timm) (1.3.0)
    Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /usr/local/lib/python3.11/dist-packages (from torchvision->timm) (11.1.0)
    Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.11/dist-packages (from jinja2->torch->timm) (3.0.2)
    Downloading nvidia_cublas_cu12-12.4.5.8-py3-none-manylinux2014_x86_64.whl (363.4 MB)
    [2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m363.4/363.4 MB[0m [31m3.7 MB/s[0m eta [36m0:00:00[0m
    [?25hDownloading nvidia_cuda_cupti_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl (13.8 MB)
    [2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m13.8/13.8 MB[0m [31m34.1 MB/s[0m eta [36m0:00:00[0m
    [?25hDownloading nvidia_cuda_nvrtc_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl (24.6 MB)
    [2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m24.6/24.6 MB[0m [31m26.1 MB/s[0m eta [36m0:00:00[0m
    [?25hDownloading nvidia_cuda_runtime_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl (883 kB)
    [2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m883.7/883.7 kB[0m [31m39.7 MB/s[0m eta [36m0:00:00[0m
    [?25hDownloading nvidia_cudnn_cu12-9.1.0.70-py3-none-manylinux2014_x86_64.whl (664.8 MB)
    [2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m664.8/664.8 MB[0m [31m2.6 MB/s[0m eta [36m0:00:00[0m
    [?25hDownloading nvidia_cufft_cu12-11.2.1.3-py3-none-manylinux2014_x86_64.whl (211.5 MB)
    [2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m211.5/211.5 MB[0m [31m5.3 MB/s[0m eta [36m0:00:00[0m
    [?25hDownloading nvidia_curand_cu12-10.3.5.147-py3-none-manylinux2014_x86_64.whl (56.3 MB)
    [2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m56.3/56.3 MB[0m [31m12.3 MB/s[0m eta [36m0:00:00[0m
    [?25hDownloading nvidia_cusolver_cu12-11.6.1.9-py3-none-manylinux2014_x86_64.whl (127.9 MB)
    [2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m127.9/127.9 MB[0m [31m7.4 MB/s[0m eta [36m0:00:00[0m
    [?25hDownloading nvidia_cusparse_cu12-12.3.1.170-py3-none-manylinux2014_x86_64.whl (207.5 MB)
    [2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m207.5/207.5 MB[0m [31m5.8 MB/s[0m eta [36m0:00:00[0m
    [?25hDownloading nvidia_nvjitlink_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl (21.1 MB)
    [2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m21.1/21.1 MB[0m [31m98.2 MB/s[0m eta [36m0:00:00[0m
    [?25hInstalling collected packages: nvidia-nvjitlink-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, nvidia-cusparse-cu12, nvidia-cudnn-cu12, nvidia-cusolver-cu12
      Attempting uninstall: nvidia-nvjitlink-cu12
        Found existing installation: nvidia-nvjitlink-cu12 12.5.82
        Uninstalling nvidia-nvjitlink-cu12-12.5.82:
          Successfully uninstalled nvidia-nvjitlink-cu12-12.5.82
      Attempting uninstall: nvidia-curand-cu12
        Found existing installation: nvidia-curand-cu12 10.3.6.82
        Uninstalling nvidia-curand-cu12-10.3.6.82:
          Successfully uninstalled nvidia-curand-cu12-10.3.6.82
      Attempting uninstall: nvidia-cufft-cu12
        Found existing installation: nvidia-cufft-cu12 11.2.3.61
        Uninstalling nvidia-cufft-cu12-11.2.3.61:
          Successfully uninstalled nvidia-cufft-cu12-11.2.3.61
      Attempting uninstall: nvidia-cuda-runtime-cu12
        Found existing installation: nvidia-cuda-runtime-cu12 12.5.82
        Uninstalling nvidia-cuda-runtime-cu12-12.5.82:
          Successfully uninstalled nvidia-cuda-runtime-cu12-12.5.82
      Attempting uninstall: nvidia-cuda-nvrtc-cu12
        Found existing installation: nvidia-cuda-nvrtc-cu12 12.5.82
        Uninstalling nvidia-cuda-nvrtc-cu12-12.5.82:
          Successfully uninstalled nvidia-cuda-nvrtc-cu12-12.5.82
      Attempting uninstall: nvidia-cuda-cupti-cu12
        Found existing installation: nvidia-cuda-cupti-cu12 12.5.82
        Uninstalling nvidia-cuda-cupti-cu12-12.5.82:
          Successfully uninstalled nvidia-cuda-cupti-cu12-12.5.82
      Attempting uninstall: nvidia-cublas-cu12
        Found existing installation: nvidia-cublas-cu12 12.5.3.2
        Uninstalling nvidia-cublas-cu12-12.5.3.2:
          Successfully uninstalled nvidia-cublas-cu12-12.5.3.2
      Attempting uninstall: nvidia-cusparse-cu12
        Found existing installation: nvidia-cusparse-cu12 12.5.1.3
        Uninstalling nvidia-cusparse-cu12-12.5.1.3:
          Successfully uninstalled nvidia-cusparse-cu12-12.5.1.3
      Attempting uninstall: nvidia-cudnn-cu12
        Found existing installation: nvidia-cudnn-cu12 9.3.0.75
        Uninstalling nvidia-cudnn-cu12-9.3.0.75:
          Successfully uninstalled nvidia-cudnn-cu12-9.3.0.75
      Attempting uninstall: nvidia-cusolver-cu12
        Found existing installation: nvidia-cusolver-cu12 11.6.3.83
        Uninstalling nvidia-cusolver-cu12-11.6.3.83:
          Successfully uninstalled nvidia-cusolver-cu12-11.6.3.83
    Successfully installed nvidia-cublas-cu12-12.4.5.8 nvidia-cuda-cupti-cu12-12.4.127 nvidia-cuda-nvrtc-cu12-12.4.127 nvidia-cuda-runtime-cu12-12.4.127 nvidia-cudnn-cu12-9.1.0.70 nvidia-cufft-cu12-11.2.1.3 nvidia-curand-cu12-10.3.5.147 nvidia-cusolver-cu12-11.6.1.9 nvidia-cusparse-cu12-12.3.1.170 nvidia-nvjitlink-cu12-12.4.127
    


```python
import torch
from PIL import Image
import requests
import json
import torchvision.transforms as T
import matplotlib.pyplot as plt
import numpy as np
import torch.nn.functional as F
from skimage.measure import find_contours
from matplotlib.patches import Polygon
from collections import defaultdict
torch.set_grad_enabled(False);
```


```python
# カラーマップ
COLORS = [[0.000, 0.447, 0.741], [0.850, 0.325, 0.098], [0.929, 0.694, 0.125],
          [0.494, 0.184, 0.556], [0.466, 0.674, 0.188], [0.301, 0.745, 0.933]]

# インプット画像の前処理
transform = T.Compose([
    T.Resize(800),
    T.ToTensor(),
    T.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
])

def box_cxcywh_to_xyxy(x):
    """
    (center_x, center_y, width, height)から(xmin, ymin, xmax, ymax)に座標変換
    """
    x_c, y_c, w, h = x.unbind(1)
    b = [(x_c - 0.5 * w), (y_c - 0.5 * h),
         (x_c + 0.5 * w), (y_c + 0.5 * h)]
    return torch.stack(b, dim=1)

def rescale_bboxes(out_bbox, size):
    """
    バウンディングボックスのリスケール
    """
    img_w, img_h = size
    b = box_cxcywh_to_xyxy(out_bbox)
    b = b * torch.tensor([img_w, img_h, img_w, img_h], dtype=torch.float32)
    return b
```


```python
def apply_mask(image, mask, color, alpha=0.5):
    """
    セグメンテーション用の領域を塗りつぶしたマスクを画像に適用

    Parameters
    ----------
    image : numpy.ndarray
      適用元の画像
    mask : tensor
      適用するマスクの領域座標
    color : list
      カラーマップ
    alpha : float
      透過値
    """
    for c in range(3):
        image[:, :, c] = np.where(mask == 1,
                                  image[:, :, c] *
                                  (1 - alpha) + alpha * color[c] * 255,
                                  image[:, :, c])
    return image
```


```python
def plot_results(pil_img, scores, boxes, labels, masks=None):
    """
    結果表示

    Parameters
    ----------
    pil_img : PIL.Image
      画像
    scores : list
      検出された物体の予測値のリスト
    boxes : list
      検出された物体のバウンディングボックス座標(center_x, center_y, width, height)のリスト
    labels : list
      検出された物体のラベルのリスト
    masks : list
      セグメンテーション用のマスクのリスト
    """
    plt.figure(figsize=(16,10))
    # PIL.Imageをnumpy.ndarrayに変換
    np_image = np.array(pil_img)
    ax = plt.gca()
    colors = COLORS * 100
    if masks is None:
        # マスクが無い場合は、len(scores)のNoneのリストで埋める
        masks = [None for _ in range(len(scores))]
    # リストの長さが違う場合、例外をスロー
    assert len(scores) == len(boxes) == len(labels) == len(masks)
    for s, (xmin, ymin, xmax, ymax), l, mask, c in zip(scores, boxes.tolist(), labels, masks, colors):
        ax.add_patch(plt.Rectangle((xmin, ymin), xmax - xmin, ymax - ymin,
                                   fill=False, color=c, linewidth=3))
        text = f'{l}: {s:0.2f}'
        ax.text(xmin, ymin, text, fontsize=15, bbox=dict(facecolor='white', alpha=0.8))

        if mask is None:
          continue
        # マスク適用
        np_image = apply_mask(np_image, mask, c)
        # マスク部分の輪郭線を描画。maskデータはバイナリなので、中間の0.5の位置で輪郭線を描画する。
        padded_mask = np.zeros((mask.shape[0] + 2, mask.shape[1] + 2), dtype=np.uint8)
        padded_mask[1:-1, 1:-1] = mask
        contours = find_contours(padded_mask, 0.5)
        for verts in contours:
            # (y, x)を(x, y)に反転
            verts = np.fliplr(verts) - 1
            # 輪郭線の内部を塗り潰し
            p = Polygon(verts, facecolor="none", edgecolor=c)
            ax.add_patch(p)

    plt.imshow(np_image)
    plt.axis('off')
    plt.show()
```

## (記事通り) 4-4. Visual Question Answering(視覚的質問応答)の推論

記事の通りに実行していきます。(コード・実行結果のみ抜粋)


```python
import json, requests
answer2id_by_type = json.load(requests.get("https://nyu.box.com/shared/static/j4rnpo8ixn6v0iznno2pim6ffj3jyaj8.json", stream=True).raw)
id2answerbytype = {}
for ans_type in answer2id_by_type.keys():
    curr_reversed_dict = {v: k for k, v in answer2id_by_type[ans_type].items()}
    id2answerbytype[ans_type] = curr_reversed_dict
```


```python
print(id2answerbytype.keys())
```

    dict_keys(['answer_rel', 'answer_obj', 'answer_global', 'answer_attr', 'answer_cat'])
    


```python
import pprint
pprint.pprint(id2answerbytype["answer_rel"])
```

    {0: 'no',
     1: 'yes',
     2: 'towel',
     3: 'man',
     4: 'backpack',
     5: 'cabinet',
     6: 'left',
     7: 'purse',
     8: 'soda',
     9: 'right',
     10: 'bag',
     11: 'bricks',
     12: 'street',
     13: 'sign',
     14: 'train',
     15: 'arrow',
     16: 'coat',
     17: 'table',
     18: 'tree',
     19: 'airplane',
     20: 'bed',
     21: 'wetsuit',
     22: 'sky',
     23: 'lettuce',
     24: 'cake',
     25: 'dog',
     26: 'horse',
     27: 'tiles',
     28: 'broccoli',
     29: 'fence',
     30: 'van',
     31: 'boats',
     32: 'bathroom',
     33: 'potato',
     34: 'giraffe',
     35: 'car',
     36: 'end table',
     37: 'wine',
     38: 'shelf',
     39: 'bus',
     40: 'tray',
     41: 'cabinets',
     42: 'truck',
     43: 'flowers',
     44: 'computer mouse',
     45: 'wall',
     46: 'radio',
     47: 'peppers',
     48: 'phone',
     49: 'lady',
     50: 'elephant',
     51: 'floor',
     52: 'skier',
     53: 'helmet',
     54: 'blender',
     55: 'cow',
     56: 'mountain',
     57: 'sofa',
     58: 'coffee table',
     59: 'grass',
     60: 'orange',
     61: 'behind',
     62: 'surfboard',
     63: 'water',
     64: 'carrots',
     65: 'tie',
     66: 'pecan',
     67: 'players',
     68: 'plate',
     69: 'umbrella',
     70: 'frisbee',
     71: 'soup',
     72: 'pizza',
     73: 'chicken',
     74: 'home plate',
     75: 'shirt',
     76: 'building',
     77: 'ground',
     78: 'sandwich',
     79: 'people',
     80: 'fire hydrant',
     81: 'trees',
     82: 'bottle',
     83: 'woman',
     84: 'papers',
     85: 'chair',
     86: 'cup',
     87: 'trailer',
     88: 'desk',
     89: 'celery',
     90: 'pen',
     91: 'bull',
     92: 'roll',
     93: 'sneakers',
     94: 'pillow',
     95: 'flamingo',
     96: 'knee pads',
     97: 'field',
     98: 'side table',
     99: 'beans',
     100: 'monitor',
     101: 'sidewalk',
     102: 'zebra',
     103: 'coffee',
     104: 'giraffes',
     105: 'cat',
     106: 'stop sign',
     107: 'laptop',
     108: 'front',
     109: 'onions',
     110: 'pan',
     111: 'tennis ball',
     112: 'hat',
     113: 'lid',
     114: 'artichokes',
     115: 'branch',
     116: 'glasses',
     117: 'spice',
     118: 'printer',
     119: 'leaves',
     120: 'cap',
     121: 'baseball bat',
     122: 'batter',
     123: 'boat',
     124: 'bear',
     125: 'rock',
     126: 'lake',
     127: 'jacket',
     128: 'train car',
     129: 'bacon',
     130: 'rice',
     131: 'bread',
     132: 'bird',
     133: 'television',
     134: 'bicycle',
     135: 'pants',
     136: 'grapes',
     137: 'beer',
     138: 'spinach',
     139: 'boy',
     140: 'child',
     141: 'tag',
     142: 'suitcase',
     143: 'cheese',
     144: 'soap',
     145: 'box',
     146: 'magnets',
     147: 'seat',
     148: 'refrigerator',
     149: 'bikini',
     150: 'oranges',
     151: 'zoo',
     152: 'keyboard',
     153: 'banana',
     154: 'skateboard',
     155: 'glass',
     156: 'hillside',
     157: 'street light',
     158: 'book',
     159: 'girl',
     160: 'bun',
     161: 'jeans',
     162: 'snow',
     163: 'stew',
     164: 'fries',
     165: 'guy',
     166: 'carriage',
     167: 'pepper',
     168: 'paper',
     169: 'carrot',
     170: 'post',
     171: 'clock',
     172: 'boots',
     173: 'ice',
     174: 'jar',
     175: 'couch',
     176: 'cars',
     177: 'clouds',
     178: 'mirror',
     179: 'onion',
     180: 'cupboard',
     181: 'suit',
     182: 'wagon',
     183: 'remote control',
     184: 'donut',
     185: 'sand',
     186: 'faucet',
     187: 'grapefruit',
     188: 'bookshelf',
     189: 'hot dog',
     190: 'drum',
     191: 'potatoes',
     192: 'bookcase',
     193: 'pole',
     194: 'meat',
     195: 'trunk',
     196: 'drawer',
     197: 'player',
     198: 'cabbage',
     199: 'words',
     200: 'rocks',
     201: 'beach',
     202: 'road',
     203: 'seagull',
     204: 'skirt',
     205: 'pastry',
     206: 'bowl',
     207: 'pasta',
     208: 'tomato',
     209: 'oven',
     210: 'cucumber',
     211: 'eggs',
     212: 'flag',
     213: 'hill',
     214: 'speaker',
     215: 'spots',
     216: 'microwave oven',
     217: 'bench',
     218: 'luggage',
     219: 'controller',
     220: 'olive',
     221: 'uniform',
     222: 'straw',
     223: 'container',
     224: 'sheep',
     225: 'gorilla',
     226: 'blanket',
     227: 'suv',
     228: 'letters',
     229: 'horses',
     230: 'sausage',
     231: 'bush',
     232: 'plant',
     233: 'shorts',
     234: 'goat',
     235: 'wii controller',
     236: 'microwave',
     237: 'bracelet',
     238: 'salad',
     239: 'doors',
     240: 'dish',
     241: 'sauce',
     242: 'teddy bear',
     243: 'ostrich',
     244: 'path',
     245: 'shrimp',
     246: 'radiator',
     247: 'tablecloth',
     248: 'chairs',
     249: 'shoe',
     250: 'window',
     251: 'baskets',
     252: 'cereal',
     253: 'computer',
     254: 'office',
     255: 'cutting board',
     256: 'donuts',
     257: 'apples',
     258: 'pickles',
     259: 'plants',
     260: 'dress',
     261: 'curtains',
     262: 'spatula',
     263: 'onion rings',
     264: 'noodles',
     265: 'scarf',
     266: 'cows',
     267: 'books',
     268: 'stove',
     269: 'envelope',
     270: 'graffiti',
     271: 'cell phone',
     272: 'school bus',
     273: 'wire',
     274: 'tablet',
     275: 'mother',
     276: 'boot',
     277: 'racket',
     278: 'cds',
     279: 'tofu',
     280: 'above',
     281: 'eye glasses',
     282: 'doorway',
     283: 'tv stand',
     284: 'doll',
     285: 'necklace',
     286: 'street sign',
     287: 'ball',
     288: 'parking meter',
     289: 'mushrooms',
     290: 'donkey',
     291: 'hay',
     292: 'blueberries',
     293: 'kettle',
     294: 'screen',
     295: 'bananas',
     296: 'dvd player',
     297: 'computer monitor',
     298: 'toaster',
     299: 'spoon',
     300: 'ducks',
     301: 'turtle',
     302: 'nightstand',
     303: 'kite',
     304: 'headphones',
     305: 'mickey mouse',
     306: 'ski',
     307: 'elephants',
     308: 'strawberries',
     309: 'bathtub',
     310: 'bike',
     311: 'number',
     312: 'camera',
     313: 'egg',
     314: 'picture',
     315: 'door',
     316: 'painting',
     317: 'barrier',
     318: 'jeep',
     319: 'dispenser',
     320: 'platform',
     321: 'powdered sugar',
     322: 'letter',
     323: 'cart',
     324: 'headband',
     325: 'canoe',
     326: 'gentleman',
     327: 'taxi',
     328: 't-shirt',
     329: 'jet',
     330: 'avocado',
     331: 'station',
     332: 'tongs',
     333: 'snack',
     334: 'bookshelves',
     335: 'roof',
     336: 'magazine',
     337: 'calculator',
     338: 'planter',
     339: 'pepperoni',
     340: 'motorcycle',
     341: 'action figure',
     342: 'binder',
     343: 'outlet',
     344: 'lamp',
     345: 'cage',
     346: 'collar',
     347: 'benches',
     348: 'garment',
     349: 'cookie',
     350: 'baby',
     351: 'office chair',
     352: 'flower',
     353: 'light switch',
     354: 'toilet paper',
     355: 'curtain',
     356: 'dishwasher',
     357: 'corn',
     358: 'seed',
     359: 'park',
     360: 'nuts',
     361: 'ocean',
     362: 'mountains',
     363: 'at the camera',
     364: 'pavement',
     365: 'ice cream',
     366: 'coleslaw',
     367: 'salad dressing',
     368: 'vase',
     369: 'dessert',
     370: 'men',
     371: 'canister',
     372: 'baseball players',
     373: 'stuffed animal',
     374: 'basket',
     375: 'lime',
     376: 'goggles',
     377: 'beverage',
     378: 'calf',
     379: 'shelves',
     380: 'napkin',
     381: 'placemat',
     382: 'skis',
     383: 'pastries',
     384: 'lamb',
     385: 'snail',
     386: 'cauliflower',
     387: 'cords',
     388: 'ice cube',
     389: 'toothbrush',
     390: 'customers',
     391: 'logs',
     392: 'cross',
     393: 'bleachers',
     394: 'raspberries',
     395: 'game controller',
     396: 'living room',
     397: 'pizza pan',
     398: 'below',
     399: 'catcher',
     400: 'toilet',
     401: 'frog',
     402: 'cherries',
     403: 'chain',
     404: 'apple',
     405: 'log',
     406: 'pilot',
     407: 'umbrellas',
     408: 'peas',
     409: 'stuffed bear',
     410: 'burner',
     411: 'chocolate',
     412: 'surfer',
     413: 'light bulbs',
     414: 'bedroom',
     415: 'tank top',
     416: 'traffic light',
     417: 'tennis',
     418: 'spectator',
     419: 'cupcakes',
     420: 'airplanes',
     421: 'sailboats',
     422: 'guitar',
     423: 'counter',
     424: 'snowboard',
     425: 'vest',
     426: 'cooler',
     427: 'bar stool',
     428: 'palm tree',
     429: 'hats',
     430: 'closet',
     431: 'speakers',
     432: 'ceiling',
     433: 'pipe',
     434: 'pineapple',
     435: 'dryer',
     436: 'sun',
     437: 'waffles',
     438: 'shore',
     439: 'desktop computer',
     440: 'cats',
     441: 'blouse',
     442: 'butterfly',
     443: 'costume',
     444: 'coffee cup',
     445: 'pot',
     446: 'coffee maker',
     447: 'towels',
     448: 'children',
     449: 'under',
     450: 'air',
     451: 'wii game',
     452: 'garland',
     453: 'dirt',
     454: 'ketchup',
     455: 'cookies',
     456: 'hard drive',
     457: 'bandana',
     458: 'runway',
     459: 'console',
     460: 'mango',
     461: 'piano',
     462: 'wires',
     463: 'tables',
     464: 'rope',
     465: 'sword',
     466: 'grape',
     467: 'bat',
     468: 'knife',
     469: 'air conditioner',
     470: 'squash',
     471: 'baseball',
     472: 'cat food',
     473: 'sticker',
     474: 'fork',
     475: 'buildings',
     476: 'locomotive',
     477: 'snacks',
     478: 'armchair',
     479: 'hallway',
     480: 'chopsticks',
     481: 'tomatoes',
     482: 'dresser',
     483: 'tower',
     484: 'trash can',
     485: 'ottoman',
     486: 'berry',
     487: 'trash bag',
     488: 'umpire',
     489: 'cupcake',
     490: 'women',
     491: 'sweet potato',
     492: 'crown',
     493: 'hot dogs',
     494: 'ice maker',
     495: 'moss',
     496: 'officer',
     497: 'ham',
     498: 'milk',
     499: 'beads',
     500: 'cupboards',
     501: 'pea',
     502: 'kitchen',
     503: 'word',
     504: 'wii',
     505: 'drawers',
     506: 'strawberry',
     507: 'goats',
     508: 'coffee pot',
     509: 'sink',
     510: 'cyclist',
     511: 'controllers',
     512: 'popcorn',
     513: 'frosting',
     514: 'duck',
     515: 'apron',
     516: 'handbag',
     517: 'countertop',
     518: 'dip',
     519: 'restroom',
     520: 'cucumbers',
     521: 'pipes',
     522: 'parrot',
     523: 'shoes',
     524: 'paddle',
     525: 'desk lamp',
     526: 'poster',
     527: 'juice',
     528: 'champagne',
     529: 'olives',
     530: 'fish',
     531: 'angry bird',
     532: 'belt',
     533: 'parking lot',
     534: 'bagels',
     535: 'bushes',
     536: 'skateboarder',
     537: 'mat',
     538: 'cowboy',
     539: 'bicycles',
     540: 'sweater',
     541: 'pineapples',
     542: 'stroller',
     543: 'dining table',
     544: 'earphones',
     545: 'rug',
     546: 'tractor',
     547: 'cherry',
     548: 'wool',
     549: 'skater',
     550: 'chickpeas',
     551: 'utensil',
     552: 'swimsuit',
     553: 'branches',
     554: 'muffin',
     555: 'beets',
     556: 'steps',
     557: 'glove',
     558: 'sushi',
     559: 'drink',
     560: 'watch',
     561: 'remote controls',
     562: 'toast',
     563: 'toppings',
     564: 'logo',
     565: 'salmon',
     566: 'socks',
     567: 'bridge',
     568: 'antenna',
     569: 'puppy',
     570: 'down',
     571: 'parsley',
     572: 'ipod',
     573: 'candies',
     574: 'island',
     575: 'birds',
     576: 'watermelon',
     577: 'xbox controller',
     578: 'lamps',
     579: 'wave',
     580: 'policeman',
     581: 'pear',
     582: 'jersey',
     583: 'water bottle',
     584: 'bags',
     585: 'blinds',
     586: 'pears',
     587: 'crowd',
     588: 'match',
     589: 'stadium',
     590: 'baking sheet',
     591: 'mozzarella',
     592: 'sheet',
     593: 'microphone',
     594: 'polar bear',
     595: 'croissant',
     596: 'minivan',
     597: 'bears',
     598: 'mushroom',
     599: 'computer desk',
     600: 'chandelier',
     601: 'packet',
     602: 'game',
     603: 'stick',
     604: 'spices',
     605: 'cable',
     606: 'bus stop',
     607: 'tissues',
     608: 'tea kettle',
     609: 'forest',
     610: 'candle holder',
     611: 'mixing bowl',
     612: 'forks',
     613: 'dragon',
     614: 'wine glass',
     615: 'fire truck',
     616: 'panda',
     617: 'wine bottle',
     618: 'toy',
     619: 'urinal',
     620: 'gas stove',
     621: 'magnet',
     622: 'porch',
     623: 'soldier',
     624: 'pans',
     625: 'outfit',
     626: 'shower',
     627: 'bucket',
     628: 'buses',
     629: 'mask',
     630: 'lemons',
     631: 'bunny',
     632: 'beds',
     633: 'artichoke',
     634: 'spear',
     635: 'tea',
     636: 'robe',
     637: 'laptops',
     638: 'gate',
     639: 'almonds',
     640: 'dishes',
     641: 'rose',
     642: 'dry-erase board',
     643: 'hotdog bun',
     644: 'owl',
     645: 'liquor',
     646: 'girls',
     647: 'heart',
     648: 'houses',
     649: 'gloves',
     650: 'pocket',
     651: 'yard',
     652: 'drinks',
     653: 'bagel',
     654: 'pajamas',
     655: 'crosswalk',
     656: 'airport',
     657: 'palm trees',
     658: 'video camera',
     659: 'statue',
     660: 'bottles',
     661: 'pedestrian',
     662: 'wines',
     663: 'luggage cart',
     664: 'pens',
     665: 'peach',
     666: 'ski lift',
     667: 'staircase',
     668: 'comforter',
     669: 'hair dryer',
     670: 'tennis balls',
     671: 'hamburger',
     672: 'medicine cabinet',
     673: 'mustard',
     674: 'aquarium',
     675: 'herbs',
     676: 'cakes',
     677: 'waffle',
     678: 'beer bottle',
     679: 'lion',
     680: 'avocados',
     681: 'foil',
     682: 'fudge',
     683: 'dumpster',
     684: 'stuffed dog',
     685: 'patio',
     686: 'heater',
     687: 'daughter',
     688: 'passenger',
     689: 'utensils',
     690: 'bubbles',
     691: 'safety jacket',
     692: 'driver',
     693: 'pigeon',
     694: 'hangar',
     695: 'garage',
     696: 'lemon',
     697: 'oil',
     698: 'boys',
     699: 'sandal',
     700: 'leaf',
     701: 'cards',
     702: 'shop',
     703: 'sandwiches',
     704: 'whisk',
     705: 'mixer',
     706: 'waste basket',
     707: 'skate park',
     708: 'ladder',
     709: 'coffee mug',
     710: 'dining room',
     711: 'ring',
     712: 'kitten',
     713: 'kites',
     714: 'silverware',
     715: 'squirrel',
     716: 'flip flop',
     717: 'power line',
     718: 'stuffed animals',
     719: 'asparagus',
     720: 'cooker',
     721: 'carpet',
     722: 'scissors',
     723: 'macaroni',
     724: 'cracker',
     725: 'stairs',
     726: 'sack',
     727: 'buns',
     728: 'brush',
     729: 'ship',
     730: 'mashed potatoes',
     731: 'cord',
     732: 'pump',
     733: 'frame',
     734: 'toiletries',
     735: 'music',
     736: 'snowboarder',
     737: 'liquid',
     738: 'candy',
     739: 'briefcase',
     740: 'artwork',
     741: 'backyard',
     742: 'soccer player',
     743: 'menu',
     744: 'food truck',
     745: 'sandals',
     746: 'upward',
     747: 'chimney',
     748: 'fans',
     749: 'deck',
     750: 'icing',
     751: 'steam',
     752: 'dough',
     753: 'spectators',
     754: 'light fixture',
     755: 'trucks',
     756: 'peaches',
     757: 'newspaper',
     758: 'crate',
     759: 'stickers',
     760: 'mouse pad',
     761: 'canopy',
     762: 'pie',
     763: 'dog food',
     764: 'tomato sauce',
     765: 'sculpture',
     766: 'chili',
     767: 'american flag',
     768: 'zebras',
     769: 'platter',
     770: 'coke',
     771: 'train station',
     772: 'roadside',
     773: 'earring',
     774: 'river',
     775: 'pitcher',
     776: 'saucer',
     777: 'soap dish',
     778: 'beet',
     779: 'cones',
     780: 'wii controllers',
     781: 'sailboat',
     782: 'bowls',
     783: 'fan',
     784: 'dressing',
     785: 'train tracks',
     786: 'tissue box',
     787: 'cables',
     788: 'gadget',
     789: 'house',
     790: 'hedge',
     791: 'pine trees',
     792: 'swimming pool',
     793: 'magazines',
     794: 'trumpet',
     795: 'turkey',
     796: 'mud',
     797: 'mugs',
     798: 'farm',
     799: 'jockey',
     800: 'dock',
     801: 'can',
     802: 'crab',
     803: 'athlete',
     804: 'marker',
     805: 'smoke',
     806: 'power lines',
     807: 'mattress',
     808: 'watermelons',
     809: 'couple',
     810: 'windows',
     811: 'soccer ball',
     812: 'desert',
     813: 'soda can',
     814: 'trains',
     815: 'net',
     816: 'bird cage',
     817: 'floor lamp',
     818: 'omelette',
     819: 'church',
     820: 'washing machine',
     821: 'granola',
     822: 'shirts',
     823: 'chains',
     824: 'ornament',
     825: 'parent',
     826: 'walkway',
     827: 'sausages',
     828: 'mayonnaise',
     829: 'seal',
     830: 'roast beef',
     831: 'water bottles',
     832: 'tools',
     833: 'puddle',
     834: 'biscuit',
     835: 'couches',
     836: 'suitcases',
     837: 'soft drink',
     838: 'berries',
     839: 'star',
     840: 'symbol',
     841: 'figure',
     842: 'chicken breast',
     843: 'hotel',
     844: 'router',
     845: 'gun',
     846: 'bee',
     847: 'rubber duck',
     848: 'containers',
     849: 'appetizers',
     850: 'raisin',
     851: 'eagle',
     852: 'topping',
     853: 'biker',
     854: 'bomb',
     855: 'decoration',
     856: 'burrito',
     857: 'pig',
     858: 'nut',
     859: 'steak',
     860: 'hammer',
     861: 'paint',
     862: 'cream cheese',
     863: 'tea pot',
     864: 'name tag',
     865: 'dugout',
     866: 'mug',
     867: 'pond',
     868: 'computers',
     869: 'ambulance',
     870: 'railroad',
     871: 'card',
     872: 'aircraft',
     873: 'pizza slice',
     874: 'seagulls',
     875: 'store',
     876: 'tent',
     877: 'sugar',
     878: 'blackberries',
     879: 'candle',
     880: 'numbers',
     881: 'sconce',
     882: 'antelope',
     883: 'bone',
     884: 'coats',
     885: 'cake stand',
     886: 'pizzas',
     887: 'bakery',
     888: 'pine tree',
     889: 'meal',
     890: 'cd',
     891: 'snowsuit',
     892: 'wolf',
     893: 'ceiling light',
     894: 'skateboards',
     895: 'gifts',
     896: 'crane',
     897: 'audience',
     898: 'balcony',
     899: 'fog',
     900: 'gravy',
     901: 'life jacket',
     902: 'heels',
     903: 'sweatshirt',
     904: 'entertainment center',
     905: 'scooter',
     906: 'pillows',
     907: 'walnuts',
     908: 'candles',
     909: 'tool',
     910: 'machine',
     911: 'eggplant',
     912: 'breakfast',
     913: 'basil',
     914: 'vines',
     915: 'passengers',
     916: 'boxes',
     917: 'muffins',
     918: 'blueberry',
     919: 'wolves',
     920: 'paper container',
     921: 'cockpit',
     922: 'skillet',
     923: 'alcohol',
     924: 'label',
     925: 'oatmeal',
     926: 'melon',
     927: 'bus driver',
     928: 'guys',
     929: 'helicopter',
     930: 'projector',
     931: 'chef',
     932: 'cloths',
     933: 'dogs',
     934: 'mound',
     935: 'jumpsuit',
     936: 'monkey',
     937: 'receipt',
     938: 'swan',
     939: 'tape',
     940: 'twigs',
     941: 'pancake',
     942: 'wallet',
     943: 'characters',
     944: 'walls',
     945: 'dvds',
     946: 'lambs',
     947: 'shopping bag',
     948: 'fur',
     949: 'soap dispenser',
     950: 'sea',
     951: 'burger',
     952: 'hair',
     953: 'flip flops',
     954: 'beer can',
     955: 'pouch',
     956: 'trash',
     957: 'package',
     958: 'flatbread',
     959: 'hurdle',
     960: 'terminal',
     961: 'hook',
     962: 'grater',
     963: 'classroom',
     964: 'rooftop',
     965: 'tunnel',
     966: 'dome',
     967: 'food',
     968: 'ladles',
     969: 'control panel',
     970: 'football',
     971: 'balls',
     972: 'melons',
     973: 'clock tower',
     974: 'stone',
     975: 'grill',
     976: 'family',
     977: 'picnic table',
     978: 'toddler',
     979: 'geese',
     980: 'beef',
     981: 'pancakes',
     982: 'cafe',
     983: 'marshmallow',
     984: 'bouquet',
     985: 'stir fry',
     986: 'pizza pie',
     987: 'up',
     988: 'bracelets',
     989: 'yogurt',
     990: 'coconut',
     991: 'charger',
     992: 'wristband',
     993: 'fireplace',
     994: 'pigeons',
     995: 'twig',
     996: 'cowboy hat',
     997: 'flags',
     998: 'gas station',
     999: 'loaf',
     1000: 'visitor',
     1001: 'drape',
     1002: 'lions',
     1003: 'comb',
     1004: 'stump',
     1005: 'tortilla',
     1006: 'bedspread',
     1007: 'games',
     1008: 'heel',
     1009: 'feeder',
     1010: 'drain',
     1011: 'bath towel',
     1012: 'video games',
     1013: 'plates',
     1014: 'produce',
     1015: 'display',
     1016: 'highway',
     1017: 'broom',
     1018: 'map',
     1019: 'salt shaker',
     1020: 'parking sign',
     1021: 'sprinkles',
     1022: 'light bulb',
     1023: 'broth',
     1024: 'whipped cream',
     1025: 'goose',
     1026: 'pizza crust',
     1027: 'photographer',
     1028: 'garden',
     1029: 'trunks',
     1030: 'lighthouse',
     1031: 'earrings',
     1032: 'rain',
     1033: 'lock',
     1034: 'biscuits',
     1035: 'electric toothbrush',
     1036: 'traffic sign',
     1037: 'tissue',
     1038: 'ropes',
     1039: 'flower pot',
     1040: 'napkins',
     1041: 'napkin dispenser',
     1042: 'raspberry',
     1043: 'drapes',
     1044: 'cameras',
     1045: 'banana peel',
     1046: 'wheelchair',
     1047: 'cups',
     1048: 'cone',
     1049: 'market',
     1050: 'figurine',
     1051: 'wildflowers',
     1052: 'blind',
     1053: 'hills',
     1054: 'dish drainer',
     1055: 'bread loaf',
     1056: 'museum',
     1057: 'picture frame',
     1058: 'notebook',
     1059: 'mall',
     1060: 'castle',
     1061: 'alligator',
     1062: 'almond',
     1063: 'face mask',
     1064: 'dress shirt',
     1065: 'meadow',
     1066: 'tangerine',
     1067: 'vacuum',
     1068: 'gym',
     1069: 'father',
     1070: 'fountain',
     1071: 'paper towel',
     1072: 'bread box',
     1073: 'soap bottle',
     1074: 'crust',
     1075: 'sponge',
     1076: 'pillowcase',
     1077: 'worker',
     1078: 'clocks',
     1079: 'pudding',
     1080: 'balloons',
     1081: 'courtyard',
     1082: 'bats',
     1083: 'crumbs',
     1084: 'rhino',
     1085: 'zucchini',
     1086: 'helmets',
     1087: 'butter',
     1088: 'battery',
     1089: 'keyboards',
     1090: 'wardrobe',
     1091: 'soccer',
     1092: 'garlic',
     1093: 'gravel',
     1094: 'pumpkin',
     1095: 'plantains',
     1096: 'back',
     1097: 'knee pad',
     1098: 'deer',
     1099: 'video game',
     1100: 'decorations',
     1101: 'cream',
     1102: 'pikachu',
     1103: 'uniforms',
     1104: 'cans',
     1105: 'cooking pot',
     1106: 'weeds',
     1107: 'money',
     1108: 'syrup',
     1109: 'bell',
     1110: 'roadway',
     1111: 'pedestrians',
     1112: 'sweet potatoes',
     1113: 'cell phones',
     1114: 'beach chair',
     1115: 'restaurant',
     1116: 'waiter',
     1117: 'trays',
     1118: 'gas pump',
     1119: 'motorcycles',
     1120: 'telephone pole',
     1121: 'intersection',
     1122: 'engineer',
     1123: 'spider',
     1124: 'workers',
     1125: 'onion ring',
     1126: 'fire',
     1127: 'mannequins',
     1128: 'toothpaste',
     1129: 'tents',
     1130: 'herd',
     1131: 'stage',
     1132: 'drawing',
     1133: 'traffic lights',
     1134: 'pasture',
     1135: 'daisy',
     1136: 'vine',
     1137: 'outlets',
     1138: 'christmas light',
     1139: 'employee',
     1140: 'monitors',
     1141: 'attic',
     1142: 'kiosk',
     1143: 'fire extinguisher',
     1144: 'satellite dish',
     1145: 'barn',
     1146: 'potato salad',
     1147: 'mailbox',
     1148: 'sea foam',
     1149: 'town',
     1150: 'elevator',
     1151: 'teddy bears',
     1152: 'tiger',
     1153: 'lego',
     1154: 'smoothie',
     1155: 'peanut butter',
     1156: 'serving dish',
     1157: 'pizza oven',
     1158: 'dish soap',
     1159: 'egg yolk',
     1160: 'paintings',
     1161: 'wine glasses',
     1162: 'leggings',
     1163: 'croissants',
     1164: 'coach',
     1165: 'table lamp',
     1166: 'nest',
     1167: 'snow pants',
     1168: 'balloon',
     1169: 'ribs',
     1170: 'pizza box',
     1171: 'shower curtain',
     1172: 'hand soap',
     1173: 'shampoo',
     1174: 'toy car',
     1175: 'buoy',
     1176: 'smoke stack',
     1177: 'hilltop',
     1178: 'orchid',
     1179: 'harbor',
     1180: 'toaster oven',
     1181: 'beneath',
     1182: 'thermometer',
     1183: 'cash register',
     1184: 'plain',
     1185: 'pots',
     1186: 'ladle',
     1187: 'camel',
     1188: 'blossoms',
     1189: 'life preserver',
     1190: 'vending machine',
     1191: 'officers',
     1192: 'hotel room',
     1193: 'step',
     1194: 'ice cubes',
     1195: 'shelter',
     1196: 'raisins',
     1197: 'fruit stand',
     1198: 'away',
     1199: 'scaffolding',
     1200: 'character',
     1201: 'mannequin',
     1202: 'notepad',
     1203: 'horse hoof',
     1204: 'lunch',
     1205: 'crackers',
     1206: 'raincoat',
     1207: 'hose',
     1208: 'pork',
     1209: 'alarm clock',
     1210: 'knives',
     1211: 'cabin',
     1212: 'shoe lace',
     1213: 'jars',
     1214: 'mangoes',
     1215: 'seafood',
     1216: 'price tag',
     1217: 'dolls',
     1218: 'polo shirt',
     1219: 'vitamins',
     1220: 'cricket',
     1221: 'sunglasses',
     1222: 'taco',
     1223: 'bug',
     1224: 'suits',
     1225: 'phones',
     1226: 'tourists',
     1227: 'forward',
     1228: 'orchids',
     1229: 'folding chair',
     1230: 'cigarette',
     1231: 'underneath',
     1232: 'shark',
     1233: 'butter knife',
     1234: 'sour cream',
     1235: 'shampoo bottle',
     1236: 'kiwi',
     1237: 'sheets',
     1238: 'garnish',
     1239: 'sticky notes',
     1240: 'cactus',
     1241: 'cereal box',
     1242: 'donkeys',
     1243: 'bikes',
     1244: 'desserts',
     1245: 'penguin',
     1246: 'tree branch',
     1247: 'pretzel',
     1248: 'casserole',
     1249: 'vests',
     1250: 'rolling pin',
     1251: 'chickens',
     1252: 'snow flakes',
     1253: 'brownie',
     1254: 'dinosaur',
     1255: 'roses',
     1256: 'blazer',
     1257: 'lawn',
     1258: 'shopping cart',
     1259: 'guacamole',
     1260: 'lemonade',
     1261: 'tuna',
     1262: 'wok',
     1263: 'potato chips',
     1264: 'performer',
     1265: 'pomegranate',
     1266: 'printers',
     1267: 'dumplings',
     1268: 'skyscraper',
     1269: 'gourd',
     1270: 'pencil',
     1271: 'water glass',
     1272: 'milkshake',
     1273: 'mountain side',
     1274: 'cheetah',
     1275: 'seeds',
     1276: 'cliff',
     1277: 'chocolate chips',
     1278: 'spoons',
     1279: 'cane',
     1280: 'keypad',
     1281: 'team',
     1282: 'meats',
     1283: 'library',
     1284: 'merchandise',
     1285: 'rabbit',
     1286: 'gown',
     1287: 'vases',
     1288: 'rice cooker',
     1289: 'baking pan',
     1290: 'blankets',
     1291: 'city',
     1292: 'swans',
     1293: 'hand dryer',
     1294: 'glaze',
     1295: 'canisters',
     1296: 'eiffel tower',
     1297: 'pizza cutter',
     1298: 'cotton dessert',
     1299: 'mexican food',
     1300: 'kittens',
     1301: 'feta cheese',
     1302: 'anchovies',
     1303: 'papaya',
     1304: 'lobster',
     1305: 'hummus',
     1306: 'sunflower',
     1307: 'bandage',
     1308: 'beverages',
     1309: 'meatballs',
     1310: 'seaweed',
     1311: 'elmo',
     1312: 'peacock',
     1313: 'antelopes',
     1314: 'boulders',
     1315: 'surfboards',
     1316: 'wallpaper',
     1317: 'cappuccino',
     1318: 'mirrors',
     1319: 'pillars',
     1320: 'food processor',
     1321: 'herb',
     1322: 'ostriches',
     1323: 'fishing pole',
     1324: 'octopus',
     1325: 'rackets',
     1326: 'ovens',
     1327: 'shield',
     1328: 'toilet brush',
     1329: 'temple',
     1330: 'kiwis',
     1331: 'cheesecake',
     1332: 'sign post',
     1333: 'parrots',
     1334: 'student',
     1335: 'french toast',
     1336: 'paper towels',
     1337: 'hamburgers',
     1338: 'carts',
     1339: 'stones',
     1340: 'serving tray',
     1341: 'moose',
     1342: 'shoe laces',
     1343: 'honey',
     1344: 'vendor',
     1345: 'stapler',
     1346: 'food container',
     1347: 'beach umbrella',
     1348: 'chinese food',
     1349: 'pita',
     1350: 'peanuts',
     1351: 'wristwatch',
     1352: 'peanut',
     1353: 'straight',
     1354: 'son',
     1355: 'knife block',
     1356: 'soda bottle',
     1357: 'stuffed bears',
     1358: 'door frame',
     1359: 'moon',
     1360: 'alien',
     1361: 'cake pan',
     1362: 'hospital',
     1363: 'panda bear',
     1364: 'toys',
     1365: 'jackets',
     1366: 'lunch box',
     1367: 'orchard',
     1368: 'cheeseburger',
     1369: 'oreo',
     1370: 'pizza boxes',
     1371: 'pencils',
     1372: 'butterflies',
     1373: 'rifle',
     1374: 'gummy bear',
     1375: 'dolphin',
     1376: 'walnut',
     1377: 'out',
     1378: 'crates',
     1379: 'olive oil',
     1380: 'shops',
     1381: 'hedges',
     1382: 'vegetable',
     1383: 'bubble',
     1384: 'bison',
     1385: 'window frame',
     1386: 'hippo',
     1387: 'sugar packets',
     1388: 'pepper shaker',
     1389: 'shuttle',
     1390: 'drawings',
     1391: 'tourist',
     1392: 'cinnamon roll',
     1393: 'christmas lights',
     1394: 'parmesan cheese',
     1395: 'brownies',
     1396: 'ravioli',
     1397: 'figurines',
     1398: 'baseball mitt',
     1399: 'bedding',
     1400: 'pretzels',
     1401: 'oak tree',
     1402: 'chopstick',
     1403: 'poodle',
     1404: 'farmer',
     1405: 'soldiers',
     1406: 'pasta salad',
     1407: 'spray bottle',
     1408: 'penguins',
     1409: 'pumpkins',
     1410: 'lounge',
     1411: 'fruit',
     1412: 'stars',
     1413: 'flames',
     1414: 'bar stools',
     1415: 'banana bunches',
     1416: 'apple logo',
     1417: 'fence post',
     1418: 'cheese cube',
     1419: 'caramel',
     1420: 'skin',
     1421: 'school',
     1422: 'rhinos',
     1423: 'sharks',
     1424: 'life jackets',
     1425: 'feathers',
     1426: 'whale',
     1427: 'sticks',
     1428: 'milk carton',
     1429: 'diaper',
     1430: 'masks',
     1431: 'entrance',
     1432: 'wig',
     1433: 'towel dispenser',
     1434: 'powder',
     1435: 'dinosaurs',
     1436: 'tags',
     1437: 'chalkboard',
     1438: 'nutella',
     1439: 'fisherman',
     1440: 'grinder',
     1441: 'tree branches',
     1442: 'flour',
     1443: 'snake',
     1444: 'banana bunch',
     1445: 'kimono',
     1446: 'mint',
     1447: 'toothbrushes',
     1448: 'customer',
     1449: 'backpacks',
     1450: 'cathedral',
     1451: 'sunflowers',
     1452: 'auditorium',
     1453: 'salt',
     1454: 'cookbook',
     1455: 'vinegar',
     1456: 'mustard bottle',
     1457: 'snakes',
     1458: 'wine bottles',
     1459: 'robot',
     1460: 'toothpicks',
     1461: 'lobby',
     1462: 'dinner',
     1463: 'hippos',
     1464: 'televisions',
     1465: 'students',
     1466: 'marina',
     1467: 'ahead',
     1468: 'garage door',
     1469: 'theater',
     1470: 'baker',
     1471: 'manhole cover',
     1472: 'gift',
     1473: 'parachute',
     1474: 'ear buds',
     1475: 'cotton candy',
     1476: 'kitchen towel',
     1477: 'swamp',
     1478: 'coin',
     1479: 'apartment building',
     1480: 'picnic tables',
     1481: 'cookie dough',
     1482: 'groceries',
     1483: 'pigs',
     1484: 'necklaces',
     1485: 'lizard',
     1486: 'obstacle',
     1487: 'cigar',
     1488: 'pandas',
     1489: 'leopard',
     1490: 'bartender',
     1491: 'village',
     1492: 'dream catcher',
     1493: 'pistachio',
     1494: 'sideways',
     1495: 'shopping center',
     1496: 'ketchup bottle',
     1497: 'baseball bats',
     1498: 'lily',
     1499: 'street lights',
     1500: 'egg carton',
     1501: 'buoys',
     1502: 'outside',
     1503: 'coconuts',
     1504: 'ornaments',
     1505: 'cages',
     1506: 'undershirt',
     1507: 'mountain peak',
     1508: 'pesto',
     1509: 'goal',
     1510: 'cranberry',
     1511: 'hearts',
     1512: 'bell tower',
     1513: 'owls',
     1514: 'salon',
     1515: 'paint brush',
     1516: 'dresses',
     1517: 'blossom',
     1518: 'beer mug',
     1519: 'cranberries',
     1520: 'athletic shoe',
     1521: 'kangaroo',
     1522: 'lipstick',
     1523: 'panda bears',
     1524: 'shaving cream',
     1525: 'cemetery',
     1526: 'shopper',
     1527: 'egg roll',
     1528: 'juice box',
     1529: 'coffee shop',
     1530: 'coffee beans',
     1531: 'ramekin',
     1532: 'blood',
     1533: 'storage box',
     1534: 'underwear',
     1535: 'beer cans',
     1536: 'taxis',
     1537: 'packages',
     1538: 'scooters',
     1539: 'waitress',
     1540: 'upwards',
     1541: 'waterfall',
     1542: 'coffee cups',
     1543: 'hair clip',
     1544: 'policemen',
     1545: 'batteries',
     1546: 'homes',
     1547: 'antennas',
     1548: 'riding boots',
     1549: 'in the mirror',
     1550: 'snow boots',
     1551: 'jewelry',
     1552: 'pizza tray',
     1553: 'sugar packet',
     1554: 'downward',
     1555: 'apartment',
     1556: 'bird house',
     1557: 'utensil holder',
     1558: 'dragons',
     1559: 'snoopy',
     1560: 'hairbrush',
     1561: 'cinnamon',
     1562: 'amusement park',
     1563: 'parachutes',
     1564: 'boar',
     1565: 'wedding',
     1566: 'supermarket',
     1567: 'tractors',
     1568: 'cookie jar',
     1569: 'blenders',
     1570: 'paper dispenser',
     1571: 'vegetables',
     1572: 'cafeteria',
     1573: 'armor',
     1574: 'towers',
     1575: 'waves',
     1576: 'pocket watch',
     1577: 'dolphins',
     1578: 'spray can',
     1579: 'nightstands',
     1580: 'elbow pad',
     1581: 'seat belt',
     1582: 'scrub brush',
     1583: 'tree leaves',
     1584: 'dragonfly',
     1585: 'mattresses',
     1586: 'appetizer',
     1587: 'dvd players',
     1588: 'swimmer',
     1589: 'stores',
     1590: 'toolbox',
     1591: 'lilies',
     1592: 'lab coat',
     1593: 'unknown'}
    


```python
# Torch Hubからモデル読込
model_qa = torch.hub.load('ashkamath/mdetr:main', 'mdetr_efficientnetB5_gqa', pretrained=True, return_postprocessor=False)
model_qa = model_qa.cuda()
model_qa.eval();
```

    /usr/local/lib/python3.11/dist-packages/torch/hub.py:330: UserWarning: You are about to download and run code from an untrusted repository. In a future release, this won't be allowed. To add the repository to your trusted list, change the command to {calling_fn}(..., trust_repo=False) and a command prompt will appear asking for an explicit confirmation of trust, or load(..., trust_repo=True), which will assume that the prompt is to be answered with 'yes'. You can also use load(..., trust_repo='check') which will only prompt for confirmation if the repo is not already trusted. This will eventually be the default behaviour
      warnings.warn(
    Downloading: "https://github.com/ashkamath/mdetr/zipball/main" to /root/.cache/torch/hub/main.zip
    /usr/local/lib/python3.11/dist-packages/timm/models/_factory.py:126: UserWarning: Mapping deprecated model name tf_efficientnet_b5_ns to current tf_efficientnet_b5.ns_jft_in1k.
      model = create_fn(
    /usr/local/lib/python3.11/dist-packages/huggingface_hub/utils/_auth.py:94: UserWarning: 
    The secret `HF_TOKEN` does not exist in your Colab secrets.
    To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.
    You will be able to reuse this secret in all of your notebooks.
    Please note that authentication is recommended but still optional to access public models or datasets.
      warnings.warn(
    


    model.safetensors:   0%|          | 0.00/122M [00:00<?, ?B/s]


    WARNING:timm.models._builder:Unexpected keys (bn2.bias, bn2.num_batches_tracked, bn2.running_mean, bn2.running_var, bn2.weight, classifier.bias, classifier.weight, conv_head.weight) found while loading pretrained weights. This may be expected if model is being adapted.
    


    tokenizer_config.json:   0%|          | 0.00/25.0 [00:00<?, ?B/s]



    vocab.json:   0%|          | 0.00/899k [00:00<?, ?B/s]



    merges.txt:   0%|          | 0.00/456k [00:00<?, ?B/s]



    tokenizer.json:   0%|          | 0.00/1.36M [00:00<?, ?B/s]



    config.json:   0%|          | 0.00/481 [00:00<?, ?B/s]


    Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
    WARNING:huggingface_hub.file_download:Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
    


    model.safetensors:   0%|          | 0.00/499M [00:00<?, ?B/s]


    Some weights of RobertaModel were not initialized from the model checkpoint at roberta-base and are newly initialized: ['pooler.dense.bias', 'pooler.dense.weight']
    You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
    Downloading: "https://zenodo.org/record/4721981/files/gqa_EB5_checkpoint.pth" to /root/.cache/torch/hub/checkpoints/gqa_EB5_checkpoint.pth
    100%|██████████| 2.54G/2.54G [03:43<00:00, 12.2MB/s]
    


    ---------------------------------------------------------------------------

    RuntimeError                              Traceback (most recent call last)

    <ipython-input-9-6faeb8f28838> in <cell line: 0>()
          1 # Torch Hubからモデル読込
    ----> 2 model_qa = torch.hub.load('ashkamath/mdetr:main', 'mdetr_efficientnetB5_gqa', pretrained=True, return_postprocessor=False)
          3 model_qa = model_qa.cuda()
          4 model_qa.eval();
    

    /usr/local/lib/python3.11/dist-packages/torch/hub.py in load(repo_or_dir, model, source, trust_repo, force_reload, verbose, skip_validation, *args, **kwargs)
        645         )
        646 
    --> 647     model = _load_local(repo_or_dir, model, *args, **kwargs)
        648     return model
        649 
    

    /usr/local/lib/python3.11/dist-packages/torch/hub.py in _load_local(hubconf_dir, model, *args, **kwargs)
        674 
        675         entry = _load_entry_from_hubconf(hub_module, model)
    --> 676         model = entry(*args, **kwargs)
        677 
        678     return model
    

    ~/.cache/torch/hub/ashkamath_mdetr_main/hubconf.py in mdetr_efficientnetB5_gqa(pretrained, return_postprocessor)
        180             url="https://zenodo.org/record/4721981/files/gqa_EB5_checkpoint.pth", map_location="cpu", check_hash=True
        181         )
    --> 182         model.load_state_dict(checkpoint["model"])
        183     if return_postprocessor:
        184         return model, PostProcess()
    

    /usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py in load_state_dict(self, state_dict, strict, assign)
       2579 
       2580         if len(error_msgs) > 0:
    -> 2581             raise RuntimeError(
       2582                 "Error(s) in loading state_dict for {}:\n\t{}".format(
       2583                     self.__class__.__name__, "\n\t".join(error_msgs)
    

    RuntimeError: Error(s) in loading state_dict for MDETR:
    	Unexpected key(s) in state_dict: "transformer.text_encoder.embeddings.position_ids". 


この「Torch Hubからモデル読込」部分でエラーが発生。

# エラーが発生、デバッグ

「Unexpected key(s) in state_dict: "transformer.text_encoder.embeddings.position_ids". 」とは、モデル定義と、学習済みパラメータのデータとの間で、整合が取れないもの(position_ids)がある模様。
以降で色々調べてみます。

## Debug: モデル側の変数を見る

学習済みパラメータの読み込み無し「pretrained=False」で、モデル定義のみ読み込んで、モデル側の該当変数の値を見てみます。


```python
model_qa = torch.hub.load('ashkamath/mdetr:main', 'mdetr_efficientnetB5_gqa', pretrained=False, return_postprocessor=False)
```

    Using cache found in /root/.cache/torch/hub/ashkamath_mdetr_main
    WARNING:timm.models._builder:Unexpected keys (bn2.bias, bn2.num_batches_tracked, bn2.running_mean, bn2.running_var, bn2.weight, classifier.bias, classifier.weight, conv_head.weight) found while loading pretrained weights. This may be expected if model is being adapted.
    Some weights of RobertaModel were not initialized from the model checkpoint at roberta-base and are newly initialized: ['pooler.dense.bias', 'pooler.dense.weight']
    You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
    


```python
model_qa.transformer.text_encoder.embeddings.position_ids
```




    tensor([[  0,   1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,  13,
              14,  15,  16,  17,  18,  19,  20,  21,  22,  23,  24,  25,  26,  27,
              28,  29,  30,  31,  32,  33,  34,  35,  36,  37,  38,  39,  40,  41,
              42,  43,  44,  45,  46,  47,  48,  49,  50,  51,  52,  53,  54,  55,
              56,  57,  58,  59,  60,  61,  62,  63,  64,  65,  66,  67,  68,  69,
              70,  71,  72,  73,  74,  75,  76,  77,  78,  79,  80,  81,  82,  83,
              84,  85,  86,  87,  88,  89,  90,  91,  92,  93,  94,  95,  96,  97,
              98,  99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111,
             112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125,
             126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139,
             140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153,
             154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167,
             168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181,
             182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195,
             196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209,
             210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223,
             224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237,
             238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251,
             252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 263, 264, 265,
             266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 277, 278, 279,
             280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290, 291, 292, 293,
             294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304, 305, 306, 307,
             308, 309, 310, 311, 312, 313, 314, 315, 316, 317, 318, 319, 320, 321,
             322, 323, 324, 325, 326, 327, 328, 329, 330, 331, 332, 333, 334, 335,
             336, 337, 338, 339, 340, 341, 342, 343, 344, 345, 346, 347, 348, 349,
             350, 351, 352, 353, 354, 355, 356, 357, 358, 359, 360, 361, 362, 363,
             364, 365, 366, 367, 368, 369, 370, 371, 372, 373, 374, 375, 376, 377,
             378, 379, 380, 381, 382, 383, 384, 385, 386, 387, 388, 389, 390, 391,
             392, 393, 394, 395, 396, 397, 398, 399, 400, 401, 402, 403, 404, 405,
             406, 407, 408, 409, 410, 411, 412, 413, 414, 415, 416, 417, 418, 419,
             420, 421, 422, 423, 424, 425, 426, 427, 428, 429, 430, 431, 432, 433,
             434, 435, 436, 437, 438, 439, 440, 441, 442, 443, 444, 445, 446, 447,
             448, 449, 450, 451, 452, 453, 454, 455, 456, 457, 458, 459, 460, 461,
             462, 463, 464, 465, 466, 467, 468, 469, 470, 471, 472, 473, 474, 475,
             476, 477, 478, 479, 480, 481, 482, 483, 484, 485, 486, 487, 488, 489,
             490, 491, 492, 493, 494, 495, 496, 497, 498, 499, 500, 501, 502, 503,
             504, 505, 506, 507, 508, 509, 510, 511, 512, 513]])



(学習パラメータではない)「tensor」型で、既に値が入っていました。

学習済みパラメータを別で読み込み、該当変数を見ると、


```python
state_dict = torch.hub.load_state_dict_from_url("https://zenodo.org/record/4721981/files/gqa_EB5_checkpoint.pth", map_location='cpu')
```


```python
state_dict.keys()
```




    dict_keys(['model', 'model_ema', 'optimizer', 'epoch', 'args'])




```python
state_dict["model"].keys()
```




    odict_keys(['transformer.encoder.layers.0.self_attn.in_proj_weight', 'transformer.encoder.layers.0.self_attn.in_proj_bias', 'transformer.encoder.layers.0.self_attn.out_proj.weight', 'transformer.encoder.layers.0.self_attn.out_proj.bias', 'transformer.encoder.layers.0.linear1.weight', 'transformer.encoder.layers.0.linear1.bias', 'transformer.encoder.layers.0.linear2.weight', 'transformer.encoder.layers.0.linear2.bias', 'transformer.encoder.layers.0.norm1.weight', 'transformer.encoder.layers.0.norm1.bias', 'transformer.encoder.layers.0.norm2.weight', 'transformer.encoder.layers.0.norm2.bias', 'transformer.encoder.layers.1.self_attn.in_proj_weight', 'transformer.encoder.layers.1.self_attn.in_proj_bias', 'transformer.encoder.layers.1.self_attn.out_proj.weight', 'transformer.encoder.layers.1.self_attn.out_proj.bias', 'transformer.encoder.layers.1.linear1.weight', 'transformer.encoder.layers.1.linear1.bias', 'transformer.encoder.layers.1.linear2.weight', 'transformer.encoder.layers.1.linear2.bias', 'transformer.encoder.layers.1.norm1.weight', 'transformer.encoder.layers.1.norm1.bias', 'transformer.encoder.layers.1.norm2.weight', 'transformer.encoder.layers.1.norm2.bias', 'transformer.encoder.layers.2.self_attn.in_proj_weight', 'transformer.encoder.layers.2.self_attn.in_proj_bias', 'transformer.encoder.layers.2.self_attn.out_proj.weight', 'transformer.encoder.layers.2.self_attn.out_proj.bias', 'transformer.encoder.layers.2.linear1.weight', 'transformer.encoder.layers.2.linear1.bias', 'transformer.encoder.layers.2.linear2.weight', 'transformer.encoder.layers.2.linear2.bias', 'transformer.encoder.layers.2.norm1.weight', 'transformer.encoder.layers.2.norm1.bias', 'transformer.encoder.layers.2.norm2.weight', 'transformer.encoder.layers.2.norm2.bias', 'transformer.encoder.layers.3.self_attn.in_proj_weight', 'transformer.encoder.layers.3.self_attn.in_proj_bias', 'transformer.encoder.layers.3.self_attn.out_proj.weight', 'transformer.encoder.layers.3.self_attn.out_proj.bias', 'transformer.encoder.layers.3.linear1.weight', 'transformer.encoder.layers.3.linear1.bias', 'transformer.encoder.layers.3.linear2.weight', 'transformer.encoder.layers.3.linear2.bias', 'transformer.encoder.layers.3.norm1.weight', 'transformer.encoder.layers.3.norm1.bias', 'transformer.encoder.layers.3.norm2.weight', 'transformer.encoder.layers.3.norm2.bias', 'transformer.encoder.layers.4.self_attn.in_proj_weight', 'transformer.encoder.layers.4.self_attn.in_proj_bias', 'transformer.encoder.layers.4.self_attn.out_proj.weight', 'transformer.encoder.layers.4.self_attn.out_proj.bias', 'transformer.encoder.layers.4.linear1.weight', 'transformer.encoder.layers.4.linear1.bias', 'transformer.encoder.layers.4.linear2.weight', 'transformer.encoder.layers.4.linear2.bias', 'transformer.encoder.layers.4.norm1.weight', 'transformer.encoder.layers.4.norm1.bias', 'transformer.encoder.layers.4.norm2.weight', 'transformer.encoder.layers.4.norm2.bias', 'transformer.encoder.layers.5.self_attn.in_proj_weight', 'transformer.encoder.layers.5.self_attn.in_proj_bias', 'transformer.encoder.layers.5.self_attn.out_proj.weight', 'transformer.encoder.layers.5.self_attn.out_proj.bias', 'transformer.encoder.layers.5.linear1.weight', 'transformer.encoder.layers.5.linear1.bias', 'transformer.encoder.layers.5.linear2.weight', 'transformer.encoder.layers.5.linear2.bias', 'transformer.encoder.layers.5.norm1.weight', 'transformer.encoder.layers.5.norm1.bias', 'transformer.encoder.layers.5.norm2.weight', 'transformer.encoder.layers.5.norm2.bias', 'transformer.decoder.layers.0.self_attn.in_proj_weight', 'transformer.decoder.layers.0.self_attn.in_proj_bias', 'transformer.decoder.layers.0.self_attn.out_proj.weight', 'transformer.decoder.layers.0.self_attn.out_proj.bias', 'transformer.decoder.layers.0.cross_attn_image.in_proj_weight', 'transformer.decoder.layers.0.cross_attn_image.in_proj_bias', 'transformer.decoder.layers.0.cross_attn_image.out_proj.weight', 'transformer.decoder.layers.0.cross_attn_image.out_proj.bias', 'transformer.decoder.layers.0.linear1.weight', 'transformer.decoder.layers.0.linear1.bias', 'transformer.decoder.layers.0.linear2.weight', 'transformer.decoder.layers.0.linear2.bias', 'transformer.decoder.layers.0.norm1.weight', 'transformer.decoder.layers.0.norm1.bias', 'transformer.decoder.layers.0.norm3.weight', 'transformer.decoder.layers.0.norm3.bias', 'transformer.decoder.layers.0.norm4.weight', 'transformer.decoder.layers.0.norm4.bias', 'transformer.decoder.layers.1.self_attn.in_proj_weight', 'transformer.decoder.layers.1.self_attn.in_proj_bias', 'transformer.decoder.layers.1.self_attn.out_proj.weight', 'transformer.decoder.layers.1.self_attn.out_proj.bias', 'transformer.decoder.layers.1.cross_attn_image.in_proj_weight', 'transformer.decoder.layers.1.cross_attn_image.in_proj_bias', 'transformer.decoder.layers.1.cross_attn_image.out_proj.weight', 'transformer.decoder.layers.1.cross_attn_image.out_proj.bias', 'transformer.decoder.layers.1.linear1.weight', 'transformer.decoder.layers.1.linear1.bias', 'transformer.decoder.layers.1.linear2.weight', 'transformer.decoder.layers.1.linear2.bias', 'transformer.decoder.layers.1.norm1.weight', 'transformer.decoder.layers.1.norm1.bias', 'transformer.decoder.layers.1.norm3.weight', 'transformer.decoder.layers.1.norm3.bias', 'transformer.decoder.layers.1.norm4.weight', 'transformer.decoder.layers.1.norm4.bias', 'transformer.decoder.layers.2.self_attn.in_proj_weight', 'transformer.decoder.layers.2.self_attn.in_proj_bias', 'transformer.decoder.layers.2.self_attn.out_proj.weight', 'transformer.decoder.layers.2.self_attn.out_proj.bias', 'transformer.decoder.layers.2.cross_attn_image.in_proj_weight', 'transformer.decoder.layers.2.cross_attn_image.in_proj_bias', 'transformer.decoder.layers.2.cross_attn_image.out_proj.weight', 'transformer.decoder.layers.2.cross_attn_image.out_proj.bias', 'transformer.decoder.layers.2.linear1.weight', 'transformer.decoder.layers.2.linear1.bias', 'transformer.decoder.layers.2.linear2.weight', 'transformer.decoder.layers.2.linear2.bias', 'transformer.decoder.layers.2.norm1.weight', 'transformer.decoder.layers.2.norm1.bias', 'transformer.decoder.layers.2.norm3.weight', 'transformer.decoder.layers.2.norm3.bias', 'transformer.decoder.layers.2.norm4.weight', 'transformer.decoder.layers.2.norm4.bias', 'transformer.decoder.layers.3.self_attn.in_proj_weight', 'transformer.decoder.layers.3.self_attn.in_proj_bias', 'transformer.decoder.layers.3.self_attn.out_proj.weight', 'transformer.decoder.layers.3.self_attn.out_proj.bias', 'transformer.decoder.layers.3.cross_attn_image.in_proj_weight', 'transformer.decoder.layers.3.cross_attn_image.in_proj_bias', 'transformer.decoder.layers.3.cross_attn_image.out_proj.weight', 'transformer.decoder.layers.3.cross_attn_image.out_proj.bias', 'transformer.decoder.layers.3.linear1.weight', 'transformer.decoder.layers.3.linear1.bias', 'transformer.decoder.layers.3.linear2.weight', 'transformer.decoder.layers.3.linear2.bias', 'transformer.decoder.layers.3.norm1.weight', 'transformer.decoder.layers.3.norm1.bias', 'transformer.decoder.layers.3.norm3.weight', 'transformer.decoder.layers.3.norm3.bias', 'transformer.decoder.layers.3.norm4.weight', 'transformer.decoder.layers.3.norm4.bias', 'transformer.decoder.layers.4.self_attn.in_proj_weight', 'transformer.decoder.layers.4.self_attn.in_proj_bias', 'transformer.decoder.layers.4.self_attn.out_proj.weight', 'transformer.decoder.layers.4.self_attn.out_proj.bias', 'transformer.decoder.layers.4.cross_attn_image.in_proj_weight', 'transformer.decoder.layers.4.cross_attn_image.in_proj_bias', 'transformer.decoder.layers.4.cross_attn_image.out_proj.weight', 'transformer.decoder.layers.4.cross_attn_image.out_proj.bias', 'transformer.decoder.layers.4.linear1.weight', 'transformer.decoder.layers.4.linear1.bias', 'transformer.decoder.layers.4.linear2.weight', 'transformer.decoder.layers.4.linear2.bias', 'transformer.decoder.layers.4.norm1.weight', 'transformer.decoder.layers.4.norm1.bias', 'transformer.decoder.layers.4.norm3.weight', 'transformer.decoder.layers.4.norm3.bias', 'transformer.decoder.layers.4.norm4.weight', 'transformer.decoder.layers.4.norm4.bias', 'transformer.decoder.layers.5.self_attn.in_proj_weight', 'transformer.decoder.layers.5.self_attn.in_proj_bias', 'transformer.decoder.layers.5.self_attn.out_proj.weight', 'transformer.decoder.layers.5.self_attn.out_proj.bias', 'transformer.decoder.layers.5.cross_attn_image.in_proj_weight', 'transformer.decoder.layers.5.cross_attn_image.in_proj_bias', 'transformer.decoder.layers.5.cross_attn_image.out_proj.weight', 'transformer.decoder.layers.5.cross_attn_image.out_proj.bias', 'transformer.decoder.layers.5.linear1.weight', 'transformer.decoder.layers.5.linear1.bias', 'transformer.decoder.layers.5.linear2.weight', 'transformer.decoder.layers.5.linear2.bias', 'transformer.decoder.layers.5.norm1.weight', 'transformer.decoder.layers.5.norm1.bias', 'transformer.decoder.layers.5.norm3.weight', 'transformer.decoder.layers.5.norm3.bias', 'transformer.decoder.layers.5.norm4.weight', 'transformer.decoder.layers.5.norm4.bias', 'transformer.decoder.norm.weight', 'transformer.decoder.norm.bias', 'transformer.text_encoder.embeddings.position_ids', 'transformer.text_encoder.embeddings.word_embeddings.weight', 'transformer.text_encoder.embeddings.position_embeddings.weight', 'transformer.text_encoder.embeddings.token_type_embeddings.weight', 'transformer.text_encoder.embeddings.LayerNorm.weight', 'transformer.text_encoder.embeddings.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.0.attention.self.query.weight', 'transformer.text_encoder.encoder.layer.0.attention.self.query.bias', 'transformer.text_encoder.encoder.layer.0.attention.self.key.weight', 'transformer.text_encoder.encoder.layer.0.attention.self.key.bias', 'transformer.text_encoder.encoder.layer.0.attention.self.value.weight', 'transformer.text_encoder.encoder.layer.0.attention.self.value.bias', 'transformer.text_encoder.encoder.layer.0.attention.output.dense.weight', 'transformer.text_encoder.encoder.layer.0.attention.output.dense.bias', 'transformer.text_encoder.encoder.layer.0.attention.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.0.attention.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.0.intermediate.dense.weight', 'transformer.text_encoder.encoder.layer.0.intermediate.dense.bias', 'transformer.text_encoder.encoder.layer.0.output.dense.weight', 'transformer.text_encoder.encoder.layer.0.output.dense.bias', 'transformer.text_encoder.encoder.layer.0.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.0.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.1.attention.self.query.weight', 'transformer.text_encoder.encoder.layer.1.attention.self.query.bias', 'transformer.text_encoder.encoder.layer.1.attention.self.key.weight', 'transformer.text_encoder.encoder.layer.1.attention.self.key.bias', 'transformer.text_encoder.encoder.layer.1.attention.self.value.weight', 'transformer.text_encoder.encoder.layer.1.attention.self.value.bias', 'transformer.text_encoder.encoder.layer.1.attention.output.dense.weight', 'transformer.text_encoder.encoder.layer.1.attention.output.dense.bias', 'transformer.text_encoder.encoder.layer.1.attention.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.1.attention.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.1.intermediate.dense.weight', 'transformer.text_encoder.encoder.layer.1.intermediate.dense.bias', 'transformer.text_encoder.encoder.layer.1.output.dense.weight', 'transformer.text_encoder.encoder.layer.1.output.dense.bias', 'transformer.text_encoder.encoder.layer.1.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.1.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.2.attention.self.query.weight', 'transformer.text_encoder.encoder.layer.2.attention.self.query.bias', 'transformer.text_encoder.encoder.layer.2.attention.self.key.weight', 'transformer.text_encoder.encoder.layer.2.attention.self.key.bias', 'transformer.text_encoder.encoder.layer.2.attention.self.value.weight', 'transformer.text_encoder.encoder.layer.2.attention.self.value.bias', 'transformer.text_encoder.encoder.layer.2.attention.output.dense.weight', 'transformer.text_encoder.encoder.layer.2.attention.output.dense.bias', 'transformer.text_encoder.encoder.layer.2.attention.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.2.attention.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.2.intermediate.dense.weight', 'transformer.text_encoder.encoder.layer.2.intermediate.dense.bias', 'transformer.text_encoder.encoder.layer.2.output.dense.weight', 'transformer.text_encoder.encoder.layer.2.output.dense.bias', 'transformer.text_encoder.encoder.layer.2.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.2.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.3.attention.self.query.weight', 'transformer.text_encoder.encoder.layer.3.attention.self.query.bias', 'transformer.text_encoder.encoder.layer.3.attention.self.key.weight', 'transformer.text_encoder.encoder.layer.3.attention.self.key.bias', 'transformer.text_encoder.encoder.layer.3.attention.self.value.weight', 'transformer.text_encoder.encoder.layer.3.attention.self.value.bias', 'transformer.text_encoder.encoder.layer.3.attention.output.dense.weight', 'transformer.text_encoder.encoder.layer.3.attention.output.dense.bias', 'transformer.text_encoder.encoder.layer.3.attention.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.3.attention.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.3.intermediate.dense.weight', 'transformer.text_encoder.encoder.layer.3.intermediate.dense.bias', 'transformer.text_encoder.encoder.layer.3.output.dense.weight', 'transformer.text_encoder.encoder.layer.3.output.dense.bias', 'transformer.text_encoder.encoder.layer.3.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.3.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.4.attention.self.query.weight', 'transformer.text_encoder.encoder.layer.4.attention.self.query.bias', 'transformer.text_encoder.encoder.layer.4.attention.self.key.weight', 'transformer.text_encoder.encoder.layer.4.attention.self.key.bias', 'transformer.text_encoder.encoder.layer.4.attention.self.value.weight', 'transformer.text_encoder.encoder.layer.4.attention.self.value.bias', 'transformer.text_encoder.encoder.layer.4.attention.output.dense.weight', 'transformer.text_encoder.encoder.layer.4.attention.output.dense.bias', 'transformer.text_encoder.encoder.layer.4.attention.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.4.attention.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.4.intermediate.dense.weight', 'transformer.text_encoder.encoder.layer.4.intermediate.dense.bias', 'transformer.text_encoder.encoder.layer.4.output.dense.weight', 'transformer.text_encoder.encoder.layer.4.output.dense.bias', 'transformer.text_encoder.encoder.layer.4.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.4.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.5.attention.self.query.weight', 'transformer.text_encoder.encoder.layer.5.attention.self.query.bias', 'transformer.text_encoder.encoder.layer.5.attention.self.key.weight', 'transformer.text_encoder.encoder.layer.5.attention.self.key.bias', 'transformer.text_encoder.encoder.layer.5.attention.self.value.weight', 'transformer.text_encoder.encoder.layer.5.attention.self.value.bias', 'transformer.text_encoder.encoder.layer.5.attention.output.dense.weight', 'transformer.text_encoder.encoder.layer.5.attention.output.dense.bias', 'transformer.text_encoder.encoder.layer.5.attention.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.5.attention.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.5.intermediate.dense.weight', 'transformer.text_encoder.encoder.layer.5.intermediate.dense.bias', 'transformer.text_encoder.encoder.layer.5.output.dense.weight', 'transformer.text_encoder.encoder.layer.5.output.dense.bias', 'transformer.text_encoder.encoder.layer.5.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.5.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.6.attention.self.query.weight', 'transformer.text_encoder.encoder.layer.6.attention.self.query.bias', 'transformer.text_encoder.encoder.layer.6.attention.self.key.weight', 'transformer.text_encoder.encoder.layer.6.attention.self.key.bias', 'transformer.text_encoder.encoder.layer.6.attention.self.value.weight', 'transformer.text_encoder.encoder.layer.6.attention.self.value.bias', 'transformer.text_encoder.encoder.layer.6.attention.output.dense.weight', 'transformer.text_encoder.encoder.layer.6.attention.output.dense.bias', 'transformer.text_encoder.encoder.layer.6.attention.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.6.attention.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.6.intermediate.dense.weight', 'transformer.text_encoder.encoder.layer.6.intermediate.dense.bias', 'transformer.text_encoder.encoder.layer.6.output.dense.weight', 'transformer.text_encoder.encoder.layer.6.output.dense.bias', 'transformer.text_encoder.encoder.layer.6.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.6.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.7.attention.self.query.weight', 'transformer.text_encoder.encoder.layer.7.attention.self.query.bias', 'transformer.text_encoder.encoder.layer.7.attention.self.key.weight', 'transformer.text_encoder.encoder.layer.7.attention.self.key.bias', 'transformer.text_encoder.encoder.layer.7.attention.self.value.weight', 'transformer.text_encoder.encoder.layer.7.attention.self.value.bias', 'transformer.text_encoder.encoder.layer.7.attention.output.dense.weight', 'transformer.text_encoder.encoder.layer.7.attention.output.dense.bias', 'transformer.text_encoder.encoder.layer.7.attention.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.7.attention.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.7.intermediate.dense.weight', 'transformer.text_encoder.encoder.layer.7.intermediate.dense.bias', 'transformer.text_encoder.encoder.layer.7.output.dense.weight', 'transformer.text_encoder.encoder.layer.7.output.dense.bias', 'transformer.text_encoder.encoder.layer.7.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.7.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.8.attention.self.query.weight', 'transformer.text_encoder.encoder.layer.8.attention.self.query.bias', 'transformer.text_encoder.encoder.layer.8.attention.self.key.weight', 'transformer.text_encoder.encoder.layer.8.attention.self.key.bias', 'transformer.text_encoder.encoder.layer.8.attention.self.value.weight', 'transformer.text_encoder.encoder.layer.8.attention.self.value.bias', 'transformer.text_encoder.encoder.layer.8.attention.output.dense.weight', 'transformer.text_encoder.encoder.layer.8.attention.output.dense.bias', 'transformer.text_encoder.encoder.layer.8.attention.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.8.attention.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.8.intermediate.dense.weight', 'transformer.text_encoder.encoder.layer.8.intermediate.dense.bias', 'transformer.text_encoder.encoder.layer.8.output.dense.weight', 'transformer.text_encoder.encoder.layer.8.output.dense.bias', 'transformer.text_encoder.encoder.layer.8.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.8.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.9.attention.self.query.weight', 'transformer.text_encoder.encoder.layer.9.attention.self.query.bias', 'transformer.text_encoder.encoder.layer.9.attention.self.key.weight', 'transformer.text_encoder.encoder.layer.9.attention.self.key.bias', 'transformer.text_encoder.encoder.layer.9.attention.self.value.weight', 'transformer.text_encoder.encoder.layer.9.attention.self.value.bias', 'transformer.text_encoder.encoder.layer.9.attention.output.dense.weight', 'transformer.text_encoder.encoder.layer.9.attention.output.dense.bias', 'transformer.text_encoder.encoder.layer.9.attention.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.9.attention.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.9.intermediate.dense.weight', 'transformer.text_encoder.encoder.layer.9.intermediate.dense.bias', 'transformer.text_encoder.encoder.layer.9.output.dense.weight', 'transformer.text_encoder.encoder.layer.9.output.dense.bias', 'transformer.text_encoder.encoder.layer.9.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.9.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.10.attention.self.query.weight', 'transformer.text_encoder.encoder.layer.10.attention.self.query.bias', 'transformer.text_encoder.encoder.layer.10.attention.self.key.weight', 'transformer.text_encoder.encoder.layer.10.attention.self.key.bias', 'transformer.text_encoder.encoder.layer.10.attention.self.value.weight', 'transformer.text_encoder.encoder.layer.10.attention.self.value.bias', 'transformer.text_encoder.encoder.layer.10.attention.output.dense.weight', 'transformer.text_encoder.encoder.layer.10.attention.output.dense.bias', 'transformer.text_encoder.encoder.layer.10.attention.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.10.attention.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.10.intermediate.dense.weight', 'transformer.text_encoder.encoder.layer.10.intermediate.dense.bias', 'transformer.text_encoder.encoder.layer.10.output.dense.weight', 'transformer.text_encoder.encoder.layer.10.output.dense.bias', 'transformer.text_encoder.encoder.layer.10.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.10.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.11.attention.self.query.weight', 'transformer.text_encoder.encoder.layer.11.attention.self.query.bias', 'transformer.text_encoder.encoder.layer.11.attention.self.key.weight', 'transformer.text_encoder.encoder.layer.11.attention.self.key.bias', 'transformer.text_encoder.encoder.layer.11.attention.self.value.weight', 'transformer.text_encoder.encoder.layer.11.attention.self.value.bias', 'transformer.text_encoder.encoder.layer.11.attention.output.dense.weight', 'transformer.text_encoder.encoder.layer.11.attention.output.dense.bias', 'transformer.text_encoder.encoder.layer.11.attention.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.11.attention.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.11.intermediate.dense.weight', 'transformer.text_encoder.encoder.layer.11.intermediate.dense.bias', 'transformer.text_encoder.encoder.layer.11.output.dense.weight', 'transformer.text_encoder.encoder.layer.11.output.dense.bias', 'transformer.text_encoder.encoder.layer.11.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.11.output.LayerNorm.bias', 'transformer.text_encoder.pooler.dense.weight', 'transformer.text_encoder.pooler.dense.bias', 'transformer.resizer.fc.weight', 'transformer.resizer.fc.bias', 'transformer.resizer.layer_norm.weight', 'transformer.resizer.layer_norm.bias', 'class_embed.weight', 'class_embed.bias', 'bbox_embed.layers.0.weight', 'bbox_embed.layers.0.bias', 'bbox_embed.layers.1.weight', 'bbox_embed.layers.1.bias', 'bbox_embed.layers.2.weight', 'bbox_embed.layers.2.bias', 'query_embed.weight', 'qa_embed.weight', 'input_proj.weight', 'input_proj.bias', 'backbone.0.body.conv_stem.weight', 'backbone.0.body.bn1.weight', 'backbone.0.body.bn1.bias', 'backbone.0.body.bn1.running_mean', 'backbone.0.body.bn1.running_var', 'backbone.0.body.blocks.0.0.conv_dw.weight', 'backbone.0.body.blocks.0.0.bn1.weight', 'backbone.0.body.blocks.0.0.bn1.bias', 'backbone.0.body.blocks.0.0.bn1.running_mean', 'backbone.0.body.blocks.0.0.bn1.running_var', 'backbone.0.body.blocks.0.0.se.conv_reduce.weight', 'backbone.0.body.blocks.0.0.se.conv_reduce.bias', 'backbone.0.body.blocks.0.0.se.conv_expand.weight', 'backbone.0.body.blocks.0.0.se.conv_expand.bias', 'backbone.0.body.blocks.0.0.conv_pw.weight', 'backbone.0.body.blocks.0.0.bn2.weight', 'backbone.0.body.blocks.0.0.bn2.bias', 'backbone.0.body.blocks.0.0.bn2.running_mean', 'backbone.0.body.blocks.0.0.bn2.running_var', 'backbone.0.body.blocks.0.1.conv_dw.weight', 'backbone.0.body.blocks.0.1.bn1.weight', 'backbone.0.body.blocks.0.1.bn1.bias', 'backbone.0.body.blocks.0.1.bn1.running_mean', 'backbone.0.body.blocks.0.1.bn1.running_var', 'backbone.0.body.blocks.0.1.se.conv_reduce.weight', 'backbone.0.body.blocks.0.1.se.conv_reduce.bias', 'backbone.0.body.blocks.0.1.se.conv_expand.weight', 'backbone.0.body.blocks.0.1.se.conv_expand.bias', 'backbone.0.body.blocks.0.1.conv_pw.weight', 'backbone.0.body.blocks.0.1.bn2.weight', 'backbone.0.body.blocks.0.1.bn2.bias', 'backbone.0.body.blocks.0.1.bn2.running_mean', 'backbone.0.body.blocks.0.1.bn2.running_var', 'backbone.0.body.blocks.0.2.conv_dw.weight', 'backbone.0.body.blocks.0.2.bn1.weight', 'backbone.0.body.blocks.0.2.bn1.bias', 'backbone.0.body.blocks.0.2.bn1.running_mean', 'backbone.0.body.blocks.0.2.bn1.running_var', 'backbone.0.body.blocks.0.2.se.conv_reduce.weight', 'backbone.0.body.blocks.0.2.se.conv_reduce.bias', 'backbone.0.body.blocks.0.2.se.conv_expand.weight', 'backbone.0.body.blocks.0.2.se.conv_expand.bias', 'backbone.0.body.blocks.0.2.conv_pw.weight', 'backbone.0.body.blocks.0.2.bn2.weight', 'backbone.0.body.blocks.0.2.bn2.bias', 'backbone.0.body.blocks.0.2.bn2.running_mean', 'backbone.0.body.blocks.0.2.bn2.running_var', 'backbone.0.body.blocks.1.0.conv_pw.weight', 'backbone.0.body.blocks.1.0.bn1.weight', 'backbone.0.body.blocks.1.0.bn1.bias', 'backbone.0.body.blocks.1.0.bn1.running_mean', 'backbone.0.body.blocks.1.0.bn1.running_var', 'backbone.0.body.blocks.1.0.conv_dw.weight', 'backbone.0.body.blocks.1.0.bn2.weight', 'backbone.0.body.blocks.1.0.bn2.bias', 'backbone.0.body.blocks.1.0.bn2.running_mean', 'backbone.0.body.blocks.1.0.bn2.running_var', 'backbone.0.body.blocks.1.0.se.conv_reduce.weight', 'backbone.0.body.blocks.1.0.se.conv_reduce.bias', 'backbone.0.body.blocks.1.0.se.conv_expand.weight', 'backbone.0.body.blocks.1.0.se.conv_expand.bias', 'backbone.0.body.blocks.1.0.conv_pwl.weight', 'backbone.0.body.blocks.1.0.bn3.weight', 'backbone.0.body.blocks.1.0.bn3.bias', 'backbone.0.body.blocks.1.0.bn3.running_mean', 'backbone.0.body.blocks.1.0.bn3.running_var', 'backbone.0.body.blocks.1.1.conv_pw.weight', 'backbone.0.body.blocks.1.1.bn1.weight', 'backbone.0.body.blocks.1.1.bn1.bias', 'backbone.0.body.blocks.1.1.bn1.running_mean', 'backbone.0.body.blocks.1.1.bn1.running_var', 'backbone.0.body.blocks.1.1.conv_dw.weight', 'backbone.0.body.blocks.1.1.bn2.weight', 'backbone.0.body.blocks.1.1.bn2.bias', 'backbone.0.body.blocks.1.1.bn2.running_mean', 'backbone.0.body.blocks.1.1.bn2.running_var', 'backbone.0.body.blocks.1.1.se.conv_reduce.weight', 'backbone.0.body.blocks.1.1.se.conv_reduce.bias', 'backbone.0.body.blocks.1.1.se.conv_expand.weight', 'backbone.0.body.blocks.1.1.se.conv_expand.bias', 'backbone.0.body.blocks.1.1.conv_pwl.weight', 'backbone.0.body.blocks.1.1.bn3.weight', 'backbone.0.body.blocks.1.1.bn3.bias', 'backbone.0.body.blocks.1.1.bn3.running_mean', 'backbone.0.body.blocks.1.1.bn3.running_var', 'backbone.0.body.blocks.1.2.conv_pw.weight', 'backbone.0.body.blocks.1.2.bn1.weight', 'backbone.0.body.blocks.1.2.bn1.bias', 'backbone.0.body.blocks.1.2.bn1.running_mean', 'backbone.0.body.blocks.1.2.bn1.running_var', 'backbone.0.body.blocks.1.2.conv_dw.weight', 'backbone.0.body.blocks.1.2.bn2.weight', 'backbone.0.body.blocks.1.2.bn2.bias', 'backbone.0.body.blocks.1.2.bn2.running_mean', 'backbone.0.body.blocks.1.2.bn2.running_var', 'backbone.0.body.blocks.1.2.se.conv_reduce.weight', 'backbone.0.body.blocks.1.2.se.conv_reduce.bias', 'backbone.0.body.blocks.1.2.se.conv_expand.weight', 'backbone.0.body.blocks.1.2.se.conv_expand.bias', 'backbone.0.body.blocks.1.2.conv_pwl.weight', 'backbone.0.body.blocks.1.2.bn3.weight', 'backbone.0.body.blocks.1.2.bn3.bias', 'backbone.0.body.blocks.1.2.bn3.running_mean', 'backbone.0.body.blocks.1.2.bn3.running_var', 'backbone.0.body.blocks.1.3.conv_pw.weight', 'backbone.0.body.blocks.1.3.bn1.weight', 'backbone.0.body.blocks.1.3.bn1.bias', 'backbone.0.body.blocks.1.3.bn1.running_mean', 'backbone.0.body.blocks.1.3.bn1.running_var', 'backbone.0.body.blocks.1.3.conv_dw.weight', 'backbone.0.body.blocks.1.3.bn2.weight', 'backbone.0.body.blocks.1.3.bn2.bias', 'backbone.0.body.blocks.1.3.bn2.running_mean', 'backbone.0.body.blocks.1.3.bn2.running_var', 'backbone.0.body.blocks.1.3.se.conv_reduce.weight', 'backbone.0.body.blocks.1.3.se.conv_reduce.bias', 'backbone.0.body.blocks.1.3.se.conv_expand.weight', 'backbone.0.body.blocks.1.3.se.conv_expand.bias', 'backbone.0.body.blocks.1.3.conv_pwl.weight', 'backbone.0.body.blocks.1.3.bn3.weight', 'backbone.0.body.blocks.1.3.bn3.bias', 'backbone.0.body.blocks.1.3.bn3.running_mean', 'backbone.0.body.blocks.1.3.bn3.running_var', 'backbone.0.body.blocks.1.4.conv_pw.weight', 'backbone.0.body.blocks.1.4.bn1.weight', 'backbone.0.body.blocks.1.4.bn1.bias', 'backbone.0.body.blocks.1.4.bn1.running_mean', 'backbone.0.body.blocks.1.4.bn1.running_var', 'backbone.0.body.blocks.1.4.conv_dw.weight', 'backbone.0.body.blocks.1.4.bn2.weight', 'backbone.0.body.blocks.1.4.bn2.bias', 'backbone.0.body.blocks.1.4.bn2.running_mean', 'backbone.0.body.blocks.1.4.bn2.running_var', 'backbone.0.body.blocks.1.4.se.conv_reduce.weight', 'backbone.0.body.blocks.1.4.se.conv_reduce.bias', 'backbone.0.body.blocks.1.4.se.conv_expand.weight', 'backbone.0.body.blocks.1.4.se.conv_expand.bias', 'backbone.0.body.blocks.1.4.conv_pwl.weight', 'backbone.0.body.blocks.1.4.bn3.weight', 'backbone.0.body.blocks.1.4.bn3.bias', 'backbone.0.body.blocks.1.4.bn3.running_mean', 'backbone.0.body.blocks.1.4.bn3.running_var', 'backbone.0.body.blocks.2.0.conv_pw.weight', 'backbone.0.body.blocks.2.0.bn1.weight', 'backbone.0.body.blocks.2.0.bn1.bias', 'backbone.0.body.blocks.2.0.bn1.running_mean', 'backbone.0.body.blocks.2.0.bn1.running_var', 'backbone.0.body.blocks.2.0.conv_dw.weight', 'backbone.0.body.blocks.2.0.bn2.weight', 'backbone.0.body.blocks.2.0.bn2.bias', 'backbone.0.body.blocks.2.0.bn2.running_mean', 'backbone.0.body.blocks.2.0.bn2.running_var', 'backbone.0.body.blocks.2.0.se.conv_reduce.weight', 'backbone.0.body.blocks.2.0.se.conv_reduce.bias', 'backbone.0.body.blocks.2.0.se.conv_expand.weight', 'backbone.0.body.blocks.2.0.se.conv_expand.bias', 'backbone.0.body.blocks.2.0.conv_pwl.weight', 'backbone.0.body.blocks.2.0.bn3.weight', 'backbone.0.body.blocks.2.0.bn3.bias', 'backbone.0.body.blocks.2.0.bn3.running_mean', 'backbone.0.body.blocks.2.0.bn3.running_var', 'backbone.0.body.blocks.2.1.conv_pw.weight', 'backbone.0.body.blocks.2.1.bn1.weight', 'backbone.0.body.blocks.2.1.bn1.bias', 'backbone.0.body.blocks.2.1.bn1.running_mean', 'backbone.0.body.blocks.2.1.bn1.running_var', 'backbone.0.body.blocks.2.1.conv_dw.weight', 'backbone.0.body.blocks.2.1.bn2.weight', 'backbone.0.body.blocks.2.1.bn2.bias', 'backbone.0.body.blocks.2.1.bn2.running_mean', 'backbone.0.body.blocks.2.1.bn2.running_var', 'backbone.0.body.blocks.2.1.se.conv_reduce.weight', 'backbone.0.body.blocks.2.1.se.conv_reduce.bias', 'backbone.0.body.blocks.2.1.se.conv_expand.weight', 'backbone.0.body.blocks.2.1.se.conv_expand.bias', 'backbone.0.body.blocks.2.1.conv_pwl.weight', 'backbone.0.body.blocks.2.1.bn3.weight', 'backbone.0.body.blocks.2.1.bn3.bias', 'backbone.0.body.blocks.2.1.bn3.running_mean', 'backbone.0.body.blocks.2.1.bn3.running_var', 'backbone.0.body.blocks.2.2.conv_pw.weight', 'backbone.0.body.blocks.2.2.bn1.weight', 'backbone.0.body.blocks.2.2.bn1.bias', 'backbone.0.body.blocks.2.2.bn1.running_mean', 'backbone.0.body.blocks.2.2.bn1.running_var', 'backbone.0.body.blocks.2.2.conv_dw.weight', 'backbone.0.body.blocks.2.2.bn2.weight', 'backbone.0.body.blocks.2.2.bn2.bias', 'backbone.0.body.blocks.2.2.bn2.running_mean', 'backbone.0.body.blocks.2.2.bn2.running_var', 'backbone.0.body.blocks.2.2.se.conv_reduce.weight', 'backbone.0.body.blocks.2.2.se.conv_reduce.bias', 'backbone.0.body.blocks.2.2.se.conv_expand.weight', 'backbone.0.body.blocks.2.2.se.conv_expand.bias', 'backbone.0.body.blocks.2.2.conv_pwl.weight', 'backbone.0.body.blocks.2.2.bn3.weight', 'backbone.0.body.blocks.2.2.bn3.bias', 'backbone.0.body.blocks.2.2.bn3.running_mean', 'backbone.0.body.blocks.2.2.bn3.running_var', 'backbone.0.body.blocks.2.3.conv_pw.weight', 'backbone.0.body.blocks.2.3.bn1.weight', 'backbone.0.body.blocks.2.3.bn1.bias', 'backbone.0.body.blocks.2.3.bn1.running_mean', 'backbone.0.body.blocks.2.3.bn1.running_var', 'backbone.0.body.blocks.2.3.conv_dw.weight', 'backbone.0.body.blocks.2.3.bn2.weight', 'backbone.0.body.blocks.2.3.bn2.bias', 'backbone.0.body.blocks.2.3.bn2.running_mean', 'backbone.0.body.blocks.2.3.bn2.running_var', 'backbone.0.body.blocks.2.3.se.conv_reduce.weight', 'backbone.0.body.blocks.2.3.se.conv_reduce.bias', 'backbone.0.body.blocks.2.3.se.conv_expand.weight', 'backbone.0.body.blocks.2.3.se.conv_expand.bias', 'backbone.0.body.blocks.2.3.conv_pwl.weight', 'backbone.0.body.blocks.2.3.bn3.weight', 'backbone.0.body.blocks.2.3.bn3.bias', 'backbone.0.body.blocks.2.3.bn3.running_mean', 'backbone.0.body.blocks.2.3.bn3.running_var', 'backbone.0.body.blocks.2.4.conv_pw.weight', 'backbone.0.body.blocks.2.4.bn1.weight', 'backbone.0.body.blocks.2.4.bn1.bias', 'backbone.0.body.blocks.2.4.bn1.running_mean', 'backbone.0.body.blocks.2.4.bn1.running_var', 'backbone.0.body.blocks.2.4.conv_dw.weight', 'backbone.0.body.blocks.2.4.bn2.weight', 'backbone.0.body.blocks.2.4.bn2.bias', 'backbone.0.body.blocks.2.4.bn2.running_mean', 'backbone.0.body.blocks.2.4.bn2.running_var', 'backbone.0.body.blocks.2.4.se.conv_reduce.weight', 'backbone.0.body.blocks.2.4.se.conv_reduce.bias', 'backbone.0.body.blocks.2.4.se.conv_expand.weight', 'backbone.0.body.blocks.2.4.se.conv_expand.bias', 'backbone.0.body.blocks.2.4.conv_pwl.weight', 'backbone.0.body.blocks.2.4.bn3.weight', 'backbone.0.body.blocks.2.4.bn3.bias', 'backbone.0.body.blocks.2.4.bn3.running_mean', 'backbone.0.body.blocks.2.4.bn3.running_var', 'backbone.0.body.blocks.3.0.conv_pw.weight', 'backbone.0.body.blocks.3.0.bn1.weight', 'backbone.0.body.blocks.3.0.bn1.bias', 'backbone.0.body.blocks.3.0.bn1.running_mean', 'backbone.0.body.blocks.3.0.bn1.running_var', 'backbone.0.body.blocks.3.0.conv_dw.weight', 'backbone.0.body.blocks.3.0.bn2.weight', 'backbone.0.body.blocks.3.0.bn2.bias', 'backbone.0.body.blocks.3.0.bn2.running_mean', 'backbone.0.body.blocks.3.0.bn2.running_var', 'backbone.0.body.blocks.3.0.se.conv_reduce.weight', 'backbone.0.body.blocks.3.0.se.conv_reduce.bias', 'backbone.0.body.blocks.3.0.se.conv_expand.weight', 'backbone.0.body.blocks.3.0.se.conv_expand.bias', 'backbone.0.body.blocks.3.0.conv_pwl.weight', 'backbone.0.body.blocks.3.0.bn3.weight', 'backbone.0.body.blocks.3.0.bn3.bias', 'backbone.0.body.blocks.3.0.bn3.running_mean', 'backbone.0.body.blocks.3.0.bn3.running_var', 'backbone.0.body.blocks.3.1.conv_pw.weight', 'backbone.0.body.blocks.3.1.bn1.weight', 'backbone.0.body.blocks.3.1.bn1.bias', 'backbone.0.body.blocks.3.1.bn1.running_mean', 'backbone.0.body.blocks.3.1.bn1.running_var', 'backbone.0.body.blocks.3.1.conv_dw.weight', 'backbone.0.body.blocks.3.1.bn2.weight', 'backbone.0.body.blocks.3.1.bn2.bias', 'backbone.0.body.blocks.3.1.bn2.running_mean', 'backbone.0.body.blocks.3.1.bn2.running_var', 'backbone.0.body.blocks.3.1.se.conv_reduce.weight', 'backbone.0.body.blocks.3.1.se.conv_reduce.bias', 'backbone.0.body.blocks.3.1.se.conv_expand.weight', 'backbone.0.body.blocks.3.1.se.conv_expand.bias', 'backbone.0.body.blocks.3.1.conv_pwl.weight', 'backbone.0.body.blocks.3.1.bn3.weight', 'backbone.0.body.blocks.3.1.bn3.bias', 'backbone.0.body.blocks.3.1.bn3.running_mean', 'backbone.0.body.blocks.3.1.bn3.running_var', 'backbone.0.body.blocks.3.2.conv_pw.weight', 'backbone.0.body.blocks.3.2.bn1.weight', 'backbone.0.body.blocks.3.2.bn1.bias', 'backbone.0.body.blocks.3.2.bn1.running_mean', 'backbone.0.body.blocks.3.2.bn1.running_var', 'backbone.0.body.blocks.3.2.conv_dw.weight', 'backbone.0.body.blocks.3.2.bn2.weight', 'backbone.0.body.blocks.3.2.bn2.bias', 'backbone.0.body.blocks.3.2.bn2.running_mean', 'backbone.0.body.blocks.3.2.bn2.running_var', 'backbone.0.body.blocks.3.2.se.conv_reduce.weight', 'backbone.0.body.blocks.3.2.se.conv_reduce.bias', 'backbone.0.body.blocks.3.2.se.conv_expand.weight', 'backbone.0.body.blocks.3.2.se.conv_expand.bias', 'backbone.0.body.blocks.3.2.conv_pwl.weight', 'backbone.0.body.blocks.3.2.bn3.weight', 'backbone.0.body.blocks.3.2.bn3.bias', 'backbone.0.body.blocks.3.2.bn3.running_mean', 'backbone.0.body.blocks.3.2.bn3.running_var', 'backbone.0.body.blocks.3.3.conv_pw.weight', 'backbone.0.body.blocks.3.3.bn1.weight', 'backbone.0.body.blocks.3.3.bn1.bias', 'backbone.0.body.blocks.3.3.bn1.running_mean', 'backbone.0.body.blocks.3.3.bn1.running_var', 'backbone.0.body.blocks.3.3.conv_dw.weight', 'backbone.0.body.blocks.3.3.bn2.weight', 'backbone.0.body.blocks.3.3.bn2.bias', 'backbone.0.body.blocks.3.3.bn2.running_mean', 'backbone.0.body.blocks.3.3.bn2.running_var', 'backbone.0.body.blocks.3.3.se.conv_reduce.weight', 'backbone.0.body.blocks.3.3.se.conv_reduce.bias', 'backbone.0.body.blocks.3.3.se.conv_expand.weight', 'backbone.0.body.blocks.3.3.se.conv_expand.bias', 'backbone.0.body.blocks.3.3.conv_pwl.weight', 'backbone.0.body.blocks.3.3.bn3.weight', 'backbone.0.body.blocks.3.3.bn3.bias', 'backbone.0.body.blocks.3.3.bn3.running_mean', 'backbone.0.body.blocks.3.3.bn3.running_var', 'backbone.0.body.blocks.3.4.conv_pw.weight', 'backbone.0.body.blocks.3.4.bn1.weight', 'backbone.0.body.blocks.3.4.bn1.bias', 'backbone.0.body.blocks.3.4.bn1.running_mean', 'backbone.0.body.blocks.3.4.bn1.running_var', 'backbone.0.body.blocks.3.4.conv_dw.weight', 'backbone.0.body.blocks.3.4.bn2.weight', 'backbone.0.body.blocks.3.4.bn2.bias', 'backbone.0.body.blocks.3.4.bn2.running_mean', 'backbone.0.body.blocks.3.4.bn2.running_var', 'backbone.0.body.blocks.3.4.se.conv_reduce.weight', 'backbone.0.body.blocks.3.4.se.conv_reduce.bias', 'backbone.0.body.blocks.3.4.se.conv_expand.weight', 'backbone.0.body.blocks.3.4.se.conv_expand.bias', 'backbone.0.body.blocks.3.4.conv_pwl.weight', 'backbone.0.body.blocks.3.4.bn3.weight', 'backbone.0.body.blocks.3.4.bn3.bias', 'backbone.0.body.blocks.3.4.bn3.running_mean', 'backbone.0.body.blocks.3.4.bn3.running_var', 'backbone.0.body.blocks.3.5.conv_pw.weight', 'backbone.0.body.blocks.3.5.bn1.weight', 'backbone.0.body.blocks.3.5.bn1.bias', 'backbone.0.body.blocks.3.5.bn1.running_mean', 'backbone.0.body.blocks.3.5.bn1.running_var', 'backbone.0.body.blocks.3.5.conv_dw.weight', 'backbone.0.body.blocks.3.5.bn2.weight', 'backbone.0.body.blocks.3.5.bn2.bias', 'backbone.0.body.blocks.3.5.bn2.running_mean', 'backbone.0.body.blocks.3.5.bn2.running_var', 'backbone.0.body.blocks.3.5.se.conv_reduce.weight', 'backbone.0.body.blocks.3.5.se.conv_reduce.bias', 'backbone.0.body.blocks.3.5.se.conv_expand.weight', 'backbone.0.body.blocks.3.5.se.conv_expand.bias', 'backbone.0.body.blocks.3.5.conv_pwl.weight', 'backbone.0.body.blocks.3.5.bn3.weight', 'backbone.0.body.blocks.3.5.bn3.bias', 'backbone.0.body.blocks.3.5.bn3.running_mean', 'backbone.0.body.blocks.3.5.bn3.running_var', 'backbone.0.body.blocks.3.6.conv_pw.weight', 'backbone.0.body.blocks.3.6.bn1.weight', 'backbone.0.body.blocks.3.6.bn1.bias', 'backbone.0.body.blocks.3.6.bn1.running_mean', 'backbone.0.body.blocks.3.6.bn1.running_var', 'backbone.0.body.blocks.3.6.conv_dw.weight', 'backbone.0.body.blocks.3.6.bn2.weight', 'backbone.0.body.blocks.3.6.bn2.bias', 'backbone.0.body.blocks.3.6.bn2.running_mean', 'backbone.0.body.blocks.3.6.bn2.running_var', 'backbone.0.body.blocks.3.6.se.conv_reduce.weight', 'backbone.0.body.blocks.3.6.se.conv_reduce.bias', 'backbone.0.body.blocks.3.6.se.conv_expand.weight', 'backbone.0.body.blocks.3.6.se.conv_expand.bias', 'backbone.0.body.blocks.3.6.conv_pwl.weight', 'backbone.0.body.blocks.3.6.bn3.weight', 'backbone.0.body.blocks.3.6.bn3.bias', 'backbone.0.body.blocks.3.6.bn3.running_mean', 'backbone.0.body.blocks.3.6.bn3.running_var', 'backbone.0.body.blocks.4.0.conv_pw.weight', 'backbone.0.body.blocks.4.0.bn1.weight', 'backbone.0.body.blocks.4.0.bn1.bias', 'backbone.0.body.blocks.4.0.bn1.running_mean', 'backbone.0.body.blocks.4.0.bn1.running_var', 'backbone.0.body.blocks.4.0.conv_dw.weight', 'backbone.0.body.blocks.4.0.bn2.weight', 'backbone.0.body.blocks.4.0.bn2.bias', 'backbone.0.body.blocks.4.0.bn2.running_mean', 'backbone.0.body.blocks.4.0.bn2.running_var', 'backbone.0.body.blocks.4.0.se.conv_reduce.weight', 'backbone.0.body.blocks.4.0.se.conv_reduce.bias', 'backbone.0.body.blocks.4.0.se.conv_expand.weight', 'backbone.0.body.blocks.4.0.se.conv_expand.bias', 'backbone.0.body.blocks.4.0.conv_pwl.weight', 'backbone.0.body.blocks.4.0.bn3.weight', 'backbone.0.body.blocks.4.0.bn3.bias', 'backbone.0.body.blocks.4.0.bn3.running_mean', 'backbone.0.body.blocks.4.0.bn3.running_var', 'backbone.0.body.blocks.4.1.conv_pw.weight', 'backbone.0.body.blocks.4.1.bn1.weight', 'backbone.0.body.blocks.4.1.bn1.bias', 'backbone.0.body.blocks.4.1.bn1.running_mean', 'backbone.0.body.blocks.4.1.bn1.running_var', 'backbone.0.body.blocks.4.1.conv_dw.weight', 'backbone.0.body.blocks.4.1.bn2.weight', 'backbone.0.body.blocks.4.1.bn2.bias', 'backbone.0.body.blocks.4.1.bn2.running_mean', 'backbone.0.body.blocks.4.1.bn2.running_var', 'backbone.0.body.blocks.4.1.se.conv_reduce.weight', 'backbone.0.body.blocks.4.1.se.conv_reduce.bias', 'backbone.0.body.blocks.4.1.se.conv_expand.weight', 'backbone.0.body.blocks.4.1.se.conv_expand.bias', 'backbone.0.body.blocks.4.1.conv_pwl.weight', 'backbone.0.body.blocks.4.1.bn3.weight', 'backbone.0.body.blocks.4.1.bn3.bias', 'backbone.0.body.blocks.4.1.bn3.running_mean', 'backbone.0.body.blocks.4.1.bn3.running_var', 'backbone.0.body.blocks.4.2.conv_pw.weight', 'backbone.0.body.blocks.4.2.bn1.weight', 'backbone.0.body.blocks.4.2.bn1.bias', 'backbone.0.body.blocks.4.2.bn1.running_mean', 'backbone.0.body.blocks.4.2.bn1.running_var', 'backbone.0.body.blocks.4.2.conv_dw.weight', 'backbone.0.body.blocks.4.2.bn2.weight', 'backbone.0.body.blocks.4.2.bn2.bias', 'backbone.0.body.blocks.4.2.bn2.running_mean', 'backbone.0.body.blocks.4.2.bn2.running_var', 'backbone.0.body.blocks.4.2.se.conv_reduce.weight', 'backbone.0.body.blocks.4.2.se.conv_reduce.bias', 'backbone.0.body.blocks.4.2.se.conv_expand.weight', 'backbone.0.body.blocks.4.2.se.conv_expand.bias', 'backbone.0.body.blocks.4.2.conv_pwl.weight', 'backbone.0.body.blocks.4.2.bn3.weight', 'backbone.0.body.blocks.4.2.bn3.bias', 'backbone.0.body.blocks.4.2.bn3.running_mean', 'backbone.0.body.blocks.4.2.bn3.running_var', 'backbone.0.body.blocks.4.3.conv_pw.weight', 'backbone.0.body.blocks.4.3.bn1.weight', 'backbone.0.body.blocks.4.3.bn1.bias', 'backbone.0.body.blocks.4.3.bn1.running_mean', 'backbone.0.body.blocks.4.3.bn1.running_var', 'backbone.0.body.blocks.4.3.conv_dw.weight', 'backbone.0.body.blocks.4.3.bn2.weight', 'backbone.0.body.blocks.4.3.bn2.bias', 'backbone.0.body.blocks.4.3.bn2.running_mean', 'backbone.0.body.blocks.4.3.bn2.running_var', 'backbone.0.body.blocks.4.3.se.conv_reduce.weight', 'backbone.0.body.blocks.4.3.se.conv_reduce.bias', 'backbone.0.body.blocks.4.3.se.conv_expand.weight', 'backbone.0.body.blocks.4.3.se.conv_expand.bias', 'backbone.0.body.blocks.4.3.conv_pwl.weight', 'backbone.0.body.blocks.4.3.bn3.weight', 'backbone.0.body.blocks.4.3.bn3.bias', 'backbone.0.body.blocks.4.3.bn3.running_mean', 'backbone.0.body.blocks.4.3.bn3.running_var', 'backbone.0.body.blocks.4.4.conv_pw.weight', 'backbone.0.body.blocks.4.4.bn1.weight', 'backbone.0.body.blocks.4.4.bn1.bias', 'backbone.0.body.blocks.4.4.bn1.running_mean', 'backbone.0.body.blocks.4.4.bn1.running_var', 'backbone.0.body.blocks.4.4.conv_dw.weight', 'backbone.0.body.blocks.4.4.bn2.weight', 'backbone.0.body.blocks.4.4.bn2.bias', 'backbone.0.body.blocks.4.4.bn2.running_mean', 'backbone.0.body.blocks.4.4.bn2.running_var', 'backbone.0.body.blocks.4.4.se.conv_reduce.weight', 'backbone.0.body.blocks.4.4.se.conv_reduce.bias', 'backbone.0.body.blocks.4.4.se.conv_expand.weight', 'backbone.0.body.blocks.4.4.se.conv_expand.bias', 'backbone.0.body.blocks.4.4.conv_pwl.weight', 'backbone.0.body.blocks.4.4.bn3.weight', 'backbone.0.body.blocks.4.4.bn3.bias', 'backbone.0.body.blocks.4.4.bn3.running_mean', 'backbone.0.body.blocks.4.4.bn3.running_var', 'backbone.0.body.blocks.4.5.conv_pw.weight', 'backbone.0.body.blocks.4.5.bn1.weight', 'backbone.0.body.blocks.4.5.bn1.bias', 'backbone.0.body.blocks.4.5.bn1.running_mean', 'backbone.0.body.blocks.4.5.bn1.running_var', 'backbone.0.body.blocks.4.5.conv_dw.weight', 'backbone.0.body.blocks.4.5.bn2.weight', 'backbone.0.body.blocks.4.5.bn2.bias', 'backbone.0.body.blocks.4.5.bn2.running_mean', 'backbone.0.body.blocks.4.5.bn2.running_var', 'backbone.0.body.blocks.4.5.se.conv_reduce.weight', 'backbone.0.body.blocks.4.5.se.conv_reduce.bias', 'backbone.0.body.blocks.4.5.se.conv_expand.weight', 'backbone.0.body.blocks.4.5.se.conv_expand.bias', 'backbone.0.body.blocks.4.5.conv_pwl.weight', 'backbone.0.body.blocks.4.5.bn3.weight', 'backbone.0.body.blocks.4.5.bn3.bias', 'backbone.0.body.blocks.4.5.bn3.running_mean', 'backbone.0.body.blocks.4.5.bn3.running_var', 'backbone.0.body.blocks.4.6.conv_pw.weight', 'backbone.0.body.blocks.4.6.bn1.weight', 'backbone.0.body.blocks.4.6.bn1.bias', 'backbone.0.body.blocks.4.6.bn1.running_mean', 'backbone.0.body.blocks.4.6.bn1.running_var', 'backbone.0.body.blocks.4.6.conv_dw.weight', 'backbone.0.body.blocks.4.6.bn2.weight', 'backbone.0.body.blocks.4.6.bn2.bias', 'backbone.0.body.blocks.4.6.bn2.running_mean', 'backbone.0.body.blocks.4.6.bn2.running_var', 'backbone.0.body.blocks.4.6.se.conv_reduce.weight', 'backbone.0.body.blocks.4.6.se.conv_reduce.bias', 'backbone.0.body.blocks.4.6.se.conv_expand.weight', 'backbone.0.body.blocks.4.6.se.conv_expand.bias', 'backbone.0.body.blocks.4.6.conv_pwl.weight', 'backbone.0.body.blocks.4.6.bn3.weight', 'backbone.0.body.blocks.4.6.bn3.bias', 'backbone.0.body.blocks.4.6.bn3.running_mean', 'backbone.0.body.blocks.4.6.bn3.running_var', 'backbone.0.body.blocks.5.0.conv_pw.weight', 'backbone.0.body.blocks.5.0.bn1.weight', 'backbone.0.body.blocks.5.0.bn1.bias', 'backbone.0.body.blocks.5.0.bn1.running_mean', 'backbone.0.body.blocks.5.0.bn1.running_var', 'backbone.0.body.blocks.5.0.conv_dw.weight', 'backbone.0.body.blocks.5.0.bn2.weight', 'backbone.0.body.blocks.5.0.bn2.bias', 'backbone.0.body.blocks.5.0.bn2.running_mean', 'backbone.0.body.blocks.5.0.bn2.running_var', 'backbone.0.body.blocks.5.0.se.conv_reduce.weight', 'backbone.0.body.blocks.5.0.se.conv_reduce.bias', 'backbone.0.body.blocks.5.0.se.conv_expand.weight', 'backbone.0.body.blocks.5.0.se.conv_expand.bias', 'backbone.0.body.blocks.5.0.conv_pwl.weight', 'backbone.0.body.blocks.5.0.bn3.weight', 'backbone.0.body.blocks.5.0.bn3.bias', 'backbone.0.body.blocks.5.0.bn3.running_mean', 'backbone.0.body.blocks.5.0.bn3.running_var', 'backbone.0.body.blocks.5.1.conv_pw.weight', 'backbone.0.body.blocks.5.1.bn1.weight', 'backbone.0.body.blocks.5.1.bn1.bias', 'backbone.0.body.blocks.5.1.bn1.running_mean', 'backbone.0.body.blocks.5.1.bn1.running_var', 'backbone.0.body.blocks.5.1.conv_dw.weight', 'backbone.0.body.blocks.5.1.bn2.weight', 'backbone.0.body.blocks.5.1.bn2.bias', 'backbone.0.body.blocks.5.1.bn2.running_mean', 'backbone.0.body.blocks.5.1.bn2.running_var', 'backbone.0.body.blocks.5.1.se.conv_reduce.weight', 'backbone.0.body.blocks.5.1.se.conv_reduce.bias', 'backbone.0.body.blocks.5.1.se.conv_expand.weight', 'backbone.0.body.blocks.5.1.se.conv_expand.bias', 'backbone.0.body.blocks.5.1.conv_pwl.weight', 'backbone.0.body.blocks.5.1.bn3.weight', 'backbone.0.body.blocks.5.1.bn3.bias', 'backbone.0.body.blocks.5.1.bn3.running_mean', 'backbone.0.body.blocks.5.1.bn3.running_var', 'backbone.0.body.blocks.5.2.conv_pw.weight', 'backbone.0.body.blocks.5.2.bn1.weight', 'backbone.0.body.blocks.5.2.bn1.bias', 'backbone.0.body.blocks.5.2.bn1.running_mean', 'backbone.0.body.blocks.5.2.bn1.running_var', 'backbone.0.body.blocks.5.2.conv_dw.weight', 'backbone.0.body.blocks.5.2.bn2.weight', 'backbone.0.body.blocks.5.2.bn2.bias', 'backbone.0.body.blocks.5.2.bn2.running_mean', 'backbone.0.body.blocks.5.2.bn2.running_var', 'backbone.0.body.blocks.5.2.se.conv_reduce.weight', 'backbone.0.body.blocks.5.2.se.conv_reduce.bias', 'backbone.0.body.blocks.5.2.se.conv_expand.weight', 'backbone.0.body.blocks.5.2.se.conv_expand.bias', 'backbone.0.body.blocks.5.2.conv_pwl.weight', 'backbone.0.body.blocks.5.2.bn3.weight', 'backbone.0.body.blocks.5.2.bn3.bias', 'backbone.0.body.blocks.5.2.bn3.running_mean', 'backbone.0.body.blocks.5.2.bn3.running_var', 'backbone.0.body.blocks.5.3.conv_pw.weight', 'backbone.0.body.blocks.5.3.bn1.weight', 'backbone.0.body.blocks.5.3.bn1.bias', 'backbone.0.body.blocks.5.3.bn1.running_mean', 'backbone.0.body.blocks.5.3.bn1.running_var', 'backbone.0.body.blocks.5.3.conv_dw.weight', 'backbone.0.body.blocks.5.3.bn2.weight', 'backbone.0.body.blocks.5.3.bn2.bias', 'backbone.0.body.blocks.5.3.bn2.running_mean', 'backbone.0.body.blocks.5.3.bn2.running_var', 'backbone.0.body.blocks.5.3.se.conv_reduce.weight', 'backbone.0.body.blocks.5.3.se.conv_reduce.bias', 'backbone.0.body.blocks.5.3.se.conv_expand.weight', 'backbone.0.body.blocks.5.3.se.conv_expand.bias', 'backbone.0.body.blocks.5.3.conv_pwl.weight', 'backbone.0.body.blocks.5.3.bn3.weight', 'backbone.0.body.blocks.5.3.bn3.bias', 'backbone.0.body.blocks.5.3.bn3.running_mean', 'backbone.0.body.blocks.5.3.bn3.running_var', 'backbone.0.body.blocks.5.4.conv_pw.weight', 'backbone.0.body.blocks.5.4.bn1.weight', 'backbone.0.body.blocks.5.4.bn1.bias', 'backbone.0.body.blocks.5.4.bn1.running_mean', 'backbone.0.body.blocks.5.4.bn1.running_var', 'backbone.0.body.blocks.5.4.conv_dw.weight', 'backbone.0.body.blocks.5.4.bn2.weight', 'backbone.0.body.blocks.5.4.bn2.bias', 'backbone.0.body.blocks.5.4.bn2.running_mean', 'backbone.0.body.blocks.5.4.bn2.running_var', 'backbone.0.body.blocks.5.4.se.conv_reduce.weight', 'backbone.0.body.blocks.5.4.se.conv_reduce.bias', 'backbone.0.body.blocks.5.4.se.conv_expand.weight', 'backbone.0.body.blocks.5.4.se.conv_expand.bias', 'backbone.0.body.blocks.5.4.conv_pwl.weight', 'backbone.0.body.blocks.5.4.bn3.weight', 'backbone.0.body.blocks.5.4.bn3.bias', 'backbone.0.body.blocks.5.4.bn3.running_mean', 'backbone.0.body.blocks.5.4.bn3.running_var', 'backbone.0.body.blocks.5.5.conv_pw.weight', 'backbone.0.body.blocks.5.5.bn1.weight', 'backbone.0.body.blocks.5.5.bn1.bias', 'backbone.0.body.blocks.5.5.bn1.running_mean', 'backbone.0.body.blocks.5.5.bn1.running_var', 'backbone.0.body.blocks.5.5.conv_dw.weight', 'backbone.0.body.blocks.5.5.bn2.weight', 'backbone.0.body.blocks.5.5.bn2.bias', 'backbone.0.body.blocks.5.5.bn2.running_mean', 'backbone.0.body.blocks.5.5.bn2.running_var', 'backbone.0.body.blocks.5.5.se.conv_reduce.weight', 'backbone.0.body.blocks.5.5.se.conv_reduce.bias', 'backbone.0.body.blocks.5.5.se.conv_expand.weight', 'backbone.0.body.blocks.5.5.se.conv_expand.bias', 'backbone.0.body.blocks.5.5.conv_pwl.weight', 'backbone.0.body.blocks.5.5.bn3.weight', 'backbone.0.body.blocks.5.5.bn3.bias', 'backbone.0.body.blocks.5.5.bn3.running_mean', 'backbone.0.body.blocks.5.5.bn3.running_var', 'backbone.0.body.blocks.5.6.conv_pw.weight', 'backbone.0.body.blocks.5.6.bn1.weight', 'backbone.0.body.blocks.5.6.bn1.bias', 'backbone.0.body.blocks.5.6.bn1.running_mean', 'backbone.0.body.blocks.5.6.bn1.running_var', 'backbone.0.body.blocks.5.6.conv_dw.weight', 'backbone.0.body.blocks.5.6.bn2.weight', 'backbone.0.body.blocks.5.6.bn2.bias', 'backbone.0.body.blocks.5.6.bn2.running_mean', 'backbone.0.body.blocks.5.6.bn2.running_var', 'backbone.0.body.blocks.5.6.se.conv_reduce.weight', 'backbone.0.body.blocks.5.6.se.conv_reduce.bias', 'backbone.0.body.blocks.5.6.se.conv_expand.weight', 'backbone.0.body.blocks.5.6.se.conv_expand.bias', 'backbone.0.body.blocks.5.6.conv_pwl.weight', 'backbone.0.body.blocks.5.6.bn3.weight', 'backbone.0.body.blocks.5.6.bn3.bias', 'backbone.0.body.blocks.5.6.bn3.running_mean', 'backbone.0.body.blocks.5.6.bn3.running_var', 'backbone.0.body.blocks.5.7.conv_pw.weight', 'backbone.0.body.blocks.5.7.bn1.weight', 'backbone.0.body.blocks.5.7.bn1.bias', 'backbone.0.body.blocks.5.7.bn1.running_mean', 'backbone.0.body.blocks.5.7.bn1.running_var', 'backbone.0.body.blocks.5.7.conv_dw.weight', 'backbone.0.body.blocks.5.7.bn2.weight', 'backbone.0.body.blocks.5.7.bn2.bias', 'backbone.0.body.blocks.5.7.bn2.running_mean', 'backbone.0.body.blocks.5.7.bn2.running_var', 'backbone.0.body.blocks.5.7.se.conv_reduce.weight', 'backbone.0.body.blocks.5.7.se.conv_reduce.bias', 'backbone.0.body.blocks.5.7.se.conv_expand.weight', 'backbone.0.body.blocks.5.7.se.conv_expand.bias', 'backbone.0.body.blocks.5.7.conv_pwl.weight', 'backbone.0.body.blocks.5.7.bn3.weight', 'backbone.0.body.blocks.5.7.bn3.bias', 'backbone.0.body.blocks.5.7.bn3.running_mean', 'backbone.0.body.blocks.5.7.bn3.running_var', 'backbone.0.body.blocks.5.8.conv_pw.weight', 'backbone.0.body.blocks.5.8.bn1.weight', 'backbone.0.body.blocks.5.8.bn1.bias', 'backbone.0.body.blocks.5.8.bn1.running_mean', 'backbone.0.body.blocks.5.8.bn1.running_var', 'backbone.0.body.blocks.5.8.conv_dw.weight', 'backbone.0.body.blocks.5.8.bn2.weight', 'backbone.0.body.blocks.5.8.bn2.bias', 'backbone.0.body.blocks.5.8.bn2.running_mean', 'backbone.0.body.blocks.5.8.bn2.running_var', 'backbone.0.body.blocks.5.8.se.conv_reduce.weight', 'backbone.0.body.blocks.5.8.se.conv_reduce.bias', 'backbone.0.body.blocks.5.8.se.conv_expand.weight', 'backbone.0.body.blocks.5.8.se.conv_expand.bias', 'backbone.0.body.blocks.5.8.conv_pwl.weight', 'backbone.0.body.blocks.5.8.bn3.weight', 'backbone.0.body.blocks.5.8.bn3.bias', 'backbone.0.body.blocks.5.8.bn3.running_mean', 'backbone.0.body.blocks.5.8.bn3.running_var', 'backbone.0.body.blocks.6.0.conv_pw.weight', 'backbone.0.body.blocks.6.0.bn1.weight', 'backbone.0.body.blocks.6.0.bn1.bias', 'backbone.0.body.blocks.6.0.bn1.running_mean', 'backbone.0.body.blocks.6.0.bn1.running_var', 'backbone.0.body.blocks.6.0.conv_dw.weight', 'backbone.0.body.blocks.6.0.bn2.weight', 'backbone.0.body.blocks.6.0.bn2.bias', 'backbone.0.body.blocks.6.0.bn2.running_mean', 'backbone.0.body.blocks.6.0.bn2.running_var', 'backbone.0.body.blocks.6.0.se.conv_reduce.weight', 'backbone.0.body.blocks.6.0.se.conv_reduce.bias', 'backbone.0.body.blocks.6.0.se.conv_expand.weight', 'backbone.0.body.blocks.6.0.se.conv_expand.bias', 'backbone.0.body.blocks.6.0.conv_pwl.weight', 'backbone.0.body.blocks.6.0.bn3.weight', 'backbone.0.body.blocks.6.0.bn3.bias', 'backbone.0.body.blocks.6.0.bn3.running_mean', 'backbone.0.body.blocks.6.0.bn3.running_var', 'backbone.0.body.blocks.6.1.conv_pw.weight', 'backbone.0.body.blocks.6.1.bn1.weight', 'backbone.0.body.blocks.6.1.bn1.bias', 'backbone.0.body.blocks.6.1.bn1.running_mean', 'backbone.0.body.blocks.6.1.bn1.running_var', 'backbone.0.body.blocks.6.1.conv_dw.weight', 'backbone.0.body.blocks.6.1.bn2.weight', 'backbone.0.body.blocks.6.1.bn2.bias', 'backbone.0.body.blocks.6.1.bn2.running_mean', 'backbone.0.body.blocks.6.1.bn2.running_var', 'backbone.0.body.blocks.6.1.se.conv_reduce.weight', 'backbone.0.body.blocks.6.1.se.conv_reduce.bias', 'backbone.0.body.blocks.6.1.se.conv_expand.weight', 'backbone.0.body.blocks.6.1.se.conv_expand.bias', 'backbone.0.body.blocks.6.1.conv_pwl.weight', 'backbone.0.body.blocks.6.1.bn3.weight', 'backbone.0.body.blocks.6.1.bn3.bias', 'backbone.0.body.blocks.6.1.bn3.running_mean', 'backbone.0.body.blocks.6.1.bn3.running_var', 'backbone.0.body.blocks.6.2.conv_pw.weight', 'backbone.0.body.blocks.6.2.bn1.weight', 'backbone.0.body.blocks.6.2.bn1.bias', 'backbone.0.body.blocks.6.2.bn1.running_mean', 'backbone.0.body.blocks.6.2.bn1.running_var', 'backbone.0.body.blocks.6.2.conv_dw.weight', 'backbone.0.body.blocks.6.2.bn2.weight', 'backbone.0.body.blocks.6.2.bn2.bias', 'backbone.0.body.blocks.6.2.bn2.running_mean', 'backbone.0.body.blocks.6.2.bn2.running_var', 'backbone.0.body.blocks.6.2.se.conv_reduce.weight', 'backbone.0.body.blocks.6.2.se.conv_reduce.bias', 'backbone.0.body.blocks.6.2.se.conv_expand.weight', 'backbone.0.body.blocks.6.2.se.conv_expand.bias', 'backbone.0.body.blocks.6.2.conv_pwl.weight', 'backbone.0.body.blocks.6.2.bn3.weight', 'backbone.0.body.blocks.6.2.bn3.bias', 'backbone.0.body.blocks.6.2.bn3.running_mean', 'backbone.0.body.blocks.6.2.bn3.running_var', 'answer_type_head.weight', 'answer_type_head.bias', 'answer_rel_head.weight', 'answer_rel_head.bias', 'answer_obj_head.weight', 'answer_obj_head.bias', 'answer_global_head.weight', 'answer_global_head.bias', 'answer_attr_head.weight', 'answer_attr_head.bias', 'answer_cat_head.weight', 'answer_cat_head.bias'])




```python
state_dict["model"]["transformer.text_encoder.embeddings.position_ids"]
```




    tensor([[  0,   1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,  13,
              14,  15,  16,  17,  18,  19,  20,  21,  22,  23,  24,  25,  26,  27,
              28,  29,  30,  31,  32,  33,  34,  35,  36,  37,  38,  39,  40,  41,
              42,  43,  44,  45,  46,  47,  48,  49,  50,  51,  52,  53,  54,  55,
              56,  57,  58,  59,  60,  61,  62,  63,  64,  65,  66,  67,  68,  69,
              70,  71,  72,  73,  74,  75,  76,  77,  78,  79,  80,  81,  82,  83,
              84,  85,  86,  87,  88,  89,  90,  91,  92,  93,  94,  95,  96,  97,
              98,  99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111,
             112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125,
             126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139,
             140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153,
             154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167,
             168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181,
             182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195,
             196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209,
             210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223,
             224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237,
             238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251,
             252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 263, 264, 265,
             266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 277, 278, 279,
             280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290, 291, 292, 293,
             294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304, 305, 306, 307,
             308, 309, 310, 311, 312, 313, 314, 315, 316, 317, 318, 319, 320, 321,
             322, 323, 324, 325, 326, 327, 328, 329, 330, 331, 332, 333, 334, 335,
             336, 337, 338, 339, 340, 341, 342, 343, 344, 345, 346, 347, 348, 349,
             350, 351, 352, 353, 354, 355, 356, 357, 358, 359, 360, 361, 362, 363,
             364, 365, 366, 367, 368, 369, 370, 371, 372, 373, 374, 375, 376, 377,
             378, 379, 380, 381, 382, 383, 384, 385, 386, 387, 388, 389, 390, 391,
             392, 393, 394, 395, 396, 397, 398, 399, 400, 401, 402, 403, 404, 405,
             406, 407, 408, 409, 410, 411, 412, 413, 414, 415, 416, 417, 418, 419,
             420, 421, 422, 423, 424, 425, 426, 427, 428, 429, 430, 431, 432, 433,
             434, 435, 436, 437, 438, 439, 440, 441, 442, 443, 444, 445, 446, 447,
             448, 449, 450, 451, 452, 453, 454, 455, 456, 457, 458, 459, 460, 461,
             462, 463, 464, 465, 466, 467, 468, 469, 470, 471, 472, 473, 474, 475,
             476, 477, 478, 479, 480, 481, 482, 483, 484, 485, 486, 487, 488, 489,
             490, 491, 492, 493, 494, 495, 496, 497, 498, 499, 500, 501, 502, 503,
             504, 505, 506, 507, 508, 509, 510, 511, 512, 513]])



学習済みパラメータ側には、学習パラメータとして「position_ids」の定義がありました。

他のパラメータの値を見ると、


```python
state_dict["model"]["transformer.text_encoder.encoder.layer.0.attention.self.query.weight"]
```




    tensor([[ 0.0024, -0.0432, -0.1008,  ...,  0.1183,  0.0301, -0.0913],
            [ 0.0030,  0.2174,  0.0536,  ...,  0.0234,  0.0858,  0.1278],
            [ 0.0735,  0.0854,  0.0009,  ..., -0.0129, -0.0278,  0.0802],
            ...,
            [-0.1637,  0.0062, -0.0330,  ...,  0.0058,  0.0643, -0.1156],
            [-0.2991,  0.0544,  0.0946,  ..., -0.0035, -0.1324,  0.0509],
            [-0.0649, -0.1165,  0.1456,  ..., -0.2730, -0.0320, -0.1308]])



他のパラメータには、データが読み込まれている模様。

変数の型を見ると、


```python
type(model_qa.transformer.text_encoder.embeddings.position_ids)
```




    torch.Tensor




```python
type(model_qa.transformer.text_encoder.encoder.layer[0].attention.self.query.weight)
```




    torch.nn.parameter.Parameter



「position_ids」は、Parameter型ではなくtensor型なので、読み込みエラーが発生していた模様。

このままモデルに学習済みパラメータを読み込むと、


```python
model_qa.load_state_dict(state_dict["model"])
```


    ---------------------------------------------------------------------------

    RuntimeError                              Traceback (most recent call last)

    <ipython-input-19-5c885fe1a66b> in <cell line: 0>()
    ----> 1 model_qa.load_state_dict(state_dict["model"])
    

    /usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py in load_state_dict(self, state_dict, strict, assign)
       2579 
       2580         if len(error_msgs) > 0:
    -> 2581             raise RuntimeError(
       2582                 "Error(s) in loading state_dict for {}:\n\t{}".format(
       2583                     self.__class__.__name__, "\n\t".join(error_msgs)
    

    RuntimeError: Error(s) in loading state_dict for MDETR:
    	Unexpected key(s) in state_dict: "transformer.text_encoder.embeddings.position_ids". 


前回同様に「position_ids」のエラーが発生。
モデル側では、「position_ids」には(学習パラメータではない)「tensor」型で既に値が入っている所に、学習済みパラメータ側では、学習パラメータとして「position_ids」の定義があり、学習パラメータとして「position_ids」を読み込もうとしたことに起因して、エラーが発生している模様。

## Debug: 「strict=False」でモデル読込

モデル定義と、学習済みパラメータのデータとの間で、整合が取れないもの(position_ids)があるので、それを無視する「strict=False」でモデルを読み込むと、


```python
import torch
torch.__version__
```




    '2.6.0+cu124'




```python
state_dict = torch.hub.load_state_dict_from_url("https://zenodo.org/record/4721981/files/gqa_EB5_checkpoint.pth", map_location='cpu')
```


```python
model_qa = torch.hub.load('ashkamath/mdetr:main', 'mdetr_efficientnetB5_gqa', pretrained=False, return_postprocessor=False)
```

    Using cache found in /root/.cache/torch/hub/ashkamath_mdetr_main
    WARNING:timm.models._builder:Unexpected keys (bn2.bias, bn2.num_batches_tracked, bn2.running_mean, bn2.running_var, bn2.weight, classifier.bias, classifier.weight, conv_head.weight) found while loading pretrained weights. This may be expected if model is being adapted.
    Some weights of RobertaModel were not initialized from the model checkpoint at roberta-base and are newly initialized: ['pooler.dense.bias', 'pooler.dense.weight']
    You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
    


```python
model_qa.load_state_dict(state_dict["model"], strict=False)
```




    _IncompatibleKeys(missing_keys=[], unexpected_keys=['transformer.text_encoder.embeddings.position_ids'])



エラーなし、読み込みは成功。

# 読込が成功したモデルを用いて、続きの推論処理を続行


```python
# Torch Hubからモデル読込
# model_qa = torch.hub.load('ashkamath/mdetr:main', 'mdetr_efficientnetB5_gqa', pretrained=True, return_postprocessor=False)
model_qa = model_qa.cuda()
model_qa.eval();
```


```python
model_qa
```




    MDETR(
      (transformer): Transformer(
        (encoder): TransformerEncoder(
          (layers): ModuleList(
            (0-5): 6 x TransformerEncoderLayer(
              (self_attn): MultiheadAttention(
                (out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)
              )
              (linear1): Linear(in_features=256, out_features=2048, bias=True)
              (dropout): Dropout(p=0.1, inplace=False)
              (linear2): Linear(in_features=2048, out_features=256, bias=True)
              (norm1): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
              (norm2): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
              (dropout1): Dropout(p=0.1, inplace=False)
              (dropout2): Dropout(p=0.1, inplace=False)
            )
          )
        )
        (decoder): TransformerDecoder(
          (layers): ModuleList(
            (0-5): 6 x TransformerDecoderLayer(
              (self_attn): MultiheadAttention(
                (out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)
              )
              (cross_attn_image): MultiheadAttention(
                (out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)
              )
              (linear1): Linear(in_features=256, out_features=2048, bias=True)
              (dropout): Dropout(p=0.1, inplace=False)
              (linear2): Linear(in_features=2048, out_features=256, bias=True)
              (norm1): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
              (norm3): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
              (norm4): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
              (dropout1): Dropout(p=0.1, inplace=False)
              (dropout3): Dropout(p=0.1, inplace=False)
              (dropout4): Dropout(p=0.1, inplace=False)
            )
          )
          (norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
        )
        (text_encoder): RobertaModel(
          (embeddings): RobertaEmbeddings(
            (word_embeddings): Embedding(50265, 768, padding_idx=1)
            (position_embeddings): Embedding(514, 768, padding_idx=1)
            (token_type_embeddings): Embedding(1, 768)
            (LayerNorm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
            (dropout): Dropout(p=0.1, inplace=False)
          )
          (encoder): RobertaEncoder(
            (layer): ModuleList(
              (0-11): 12 x RobertaLayer(
                (attention): RobertaAttention(
                  (self): RobertaSdpaSelfAttention(
                    (query): Linear(in_features=768, out_features=768, bias=True)
                    (key): Linear(in_features=768, out_features=768, bias=True)
                    (value): Linear(in_features=768, out_features=768, bias=True)
                    (dropout): Dropout(p=0.1, inplace=False)
                  )
                  (output): RobertaSelfOutput(
                    (dense): Linear(in_features=768, out_features=768, bias=True)
                    (LayerNorm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
                    (dropout): Dropout(p=0.1, inplace=False)
                  )
                )
                (intermediate): RobertaIntermediate(
                  (dense): Linear(in_features=768, out_features=3072, bias=True)
                  (intermediate_act_fn): GELUActivation()
                )
                (output): RobertaOutput(
                  (dense): Linear(in_features=3072, out_features=768, bias=True)
                  (LayerNorm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
                  (dropout): Dropout(p=0.1, inplace=False)
                )
              )
            )
          )
          (pooler): RobertaPooler(
            (dense): Linear(in_features=768, out_features=768, bias=True)
            (activation): Tanh()
          )
        )
        (resizer): FeatureResizer(
          (fc): Linear(in_features=768, out_features=256, bias=True)
          (layer_norm): LayerNorm((256,), eps=1e-12, elementwise_affine=True)
          (dropout): Dropout(p=0.1, inplace=False)
        )
      )
      (class_embed): Linear(in_features=256, out_features=256, bias=True)
      (bbox_embed): MLP(
        (layers): ModuleList(
          (0-1): 2 x Linear(in_features=256, out_features=256, bias=True)
          (2): Linear(in_features=256, out_features=4, bias=True)
        )
      )
      (query_embed): Embedding(100, 256)
      (qa_embed): Embedding(6, 256)
      (input_proj): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1))
      (backbone): Joiner(
        (0): TimmBackbone(
          (body): EfficientNetFeatures(
            (conv_stem): Conv2dSame(3, 48, kernel_size=(3, 3), stride=(2, 2), bias=False)
            (bn1): FrozenBatchNorm2d()
            (blocks): Sequential(
              (0): Sequential(
                (0): DepthwiseSeparableConv(
                  (conv_dw): Conv2d(48, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=48, bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(48, 12, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(12, 48, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pw): Conv2d(48, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (1): DepthwiseSeparableConv(
                  (conv_dw): Conv2d(24, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=24, bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(24, 6, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(6, 24, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pw): Conv2d(24, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (2): DepthwiseSeparableConv(
                  (conv_dw): Conv2d(24, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=24, bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(24, 6, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(6, 24, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pw): Conv2d(24, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
              )
              (1): Sequential(
                (0): InvertedResidual(
                  (conv_pw): Conv2d(24, 144, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2dSame(144, 144, kernel_size=(3, 3), stride=(2, 2), groups=144, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(144, 6, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(6, 144, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(144, 40, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (1): InvertedResidual(
                  (conv_pw): Conv2d(40, 240, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(240, 240, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=240, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(240, 10, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(10, 240, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(240, 40, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (2): InvertedResidual(
                  (conv_pw): Conv2d(40, 240, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(240, 240, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=240, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(240, 10, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(10, 240, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(240, 40, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (3): InvertedResidual(
                  (conv_pw): Conv2d(40, 240, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(240, 240, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=240, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(240, 10, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(10, 240, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(240, 40, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (4): InvertedResidual(
                  (conv_pw): Conv2d(40, 240, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(240, 240, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=240, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(240, 10, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(10, 240, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(240, 40, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
              )
              (2): Sequential(
                (0): InvertedResidual(
                  (conv_pw): Conv2d(40, 240, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2dSame(240, 240, kernel_size=(5, 5), stride=(2, 2), groups=240, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(240, 10, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(10, 240, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(240, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (1): InvertedResidual(
                  (conv_pw): Conv2d(64, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(384, 384, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=384, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(384, 16, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(16, 384, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(384, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (2): InvertedResidual(
                  (conv_pw): Conv2d(64, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(384, 384, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=384, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(384, 16, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(16, 384, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(384, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (3): InvertedResidual(
                  (conv_pw): Conv2d(64, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(384, 384, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=384, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(384, 16, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(16, 384, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(384, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (4): InvertedResidual(
                  (conv_pw): Conv2d(64, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(384, 384, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=384, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(384, 16, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(16, 384, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(384, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
              )
              (3): Sequential(
                (0): InvertedResidual(
                  (conv_pw): Conv2d(64, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2dSame(384, 384, kernel_size=(3, 3), stride=(2, 2), groups=384, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(384, 16, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(16, 384, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(384, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (1): InvertedResidual(
                  (conv_pw): Conv2d(128, 768, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(768, 768, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=768, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(768, 32, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(32, 768, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(768, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (2): InvertedResidual(
                  (conv_pw): Conv2d(128, 768, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(768, 768, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=768, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(768, 32, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(32, 768, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(768, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (3): InvertedResidual(
                  (conv_pw): Conv2d(128, 768, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(768, 768, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=768, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(768, 32, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(32, 768, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(768, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (4): InvertedResidual(
                  (conv_pw): Conv2d(128, 768, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(768, 768, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=768, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(768, 32, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(32, 768, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(768, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (5): InvertedResidual(
                  (conv_pw): Conv2d(128, 768, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(768, 768, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=768, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(768, 32, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(32, 768, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(768, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (6): InvertedResidual(
                  (conv_pw): Conv2d(128, 768, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(768, 768, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=768, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(768, 32, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(32, 768, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(768, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
              )
              (4): Sequential(
                (0): InvertedResidual(
                  (conv_pw): Conv2d(128, 768, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(768, 768, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=768, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(768, 32, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(32, 768, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(768, 176, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (1): InvertedResidual(
                  (conv_pw): Conv2d(176, 1056, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(1056, 1056, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1056, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(1056, 44, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(44, 1056, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(1056, 176, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (2): InvertedResidual(
                  (conv_pw): Conv2d(176, 1056, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(1056, 1056, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1056, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(1056, 44, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(44, 1056, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(1056, 176, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (3): InvertedResidual(
                  (conv_pw): Conv2d(176, 1056, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(1056, 1056, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1056, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(1056, 44, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(44, 1056, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(1056, 176, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (4): InvertedResidual(
                  (conv_pw): Conv2d(176, 1056, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(1056, 1056, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1056, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(1056, 44, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(44, 1056, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(1056, 176, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (5): InvertedResidual(
                  (conv_pw): Conv2d(176, 1056, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(1056, 1056, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1056, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(1056, 44, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(44, 1056, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(1056, 176, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (6): InvertedResidual(
                  (conv_pw): Conv2d(176, 1056, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(1056, 1056, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1056, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(1056, 44, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(44, 1056, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(1056, 176, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
              )
              (5): Sequential(
                (0): InvertedResidual(
                  (conv_pw): Conv2d(176, 1056, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2dSame(1056, 1056, kernel_size=(5, 5), stride=(2, 2), groups=1056, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(1056, 44, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(44, 1056, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(1056, 304, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (1): InvertedResidual(
                  (conv_pw): Conv2d(304, 1824, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(1824, 1824, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1824, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(1824, 76, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(76, 1824, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(1824, 304, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (2): InvertedResidual(
                  (conv_pw): Conv2d(304, 1824, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(1824, 1824, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1824, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(1824, 76, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(76, 1824, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(1824, 304, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (3): InvertedResidual(
                  (conv_pw): Conv2d(304, 1824, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(1824, 1824, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1824, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(1824, 76, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(76, 1824, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(1824, 304, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (4): InvertedResidual(
                  (conv_pw): Conv2d(304, 1824, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(1824, 1824, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1824, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(1824, 76, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(76, 1824, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(1824, 304, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (5): InvertedResidual(
                  (conv_pw): Conv2d(304, 1824, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(1824, 1824, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1824, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(1824, 76, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(76, 1824, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(1824, 304, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (6): InvertedResidual(
                  (conv_pw): Conv2d(304, 1824, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(1824, 1824, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1824, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(1824, 76, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(76, 1824, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(1824, 304, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (7): InvertedResidual(
                  (conv_pw): Conv2d(304, 1824, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(1824, 1824, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1824, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(1824, 76, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(76, 1824, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(1824, 304, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (8): InvertedResidual(
                  (conv_pw): Conv2d(304, 1824, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(1824, 1824, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1824, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(1824, 76, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(76, 1824, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(1824, 304, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
              )
              (6): Sequential(
                (0): InvertedResidual(
                  (conv_pw): Conv2d(304, 1824, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(1824, 1824, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=1824, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(1824, 76, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(76, 1824, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(1824, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (1): InvertedResidual(
                  (conv_pw): Conv2d(512, 3072, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(3072, 3072, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=3072, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(3072, 128, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(128, 3072, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(3072, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
                (2): InvertedResidual(
                  (conv_pw): Conv2d(512, 3072, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn1): FrozenBatchNorm2d()
                  (conv_dw): Conv2d(3072, 3072, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=3072, bias=False)
                  (bn2): FrozenBatchNorm2d()
                  (aa): Identity()
                  (se): SqueezeExcite(
                    (conv_reduce): Conv2d(3072, 128, kernel_size=(1, 1), stride=(1, 1))
                    (act1): SiLU(inplace=True)
                    (conv_expand): Conv2d(128, 3072, kernel_size=(1, 1), stride=(1, 1))
                    (gate): Sigmoid()
                  )
                  (conv_pwl): Conv2d(3072, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
                  (bn3): FrozenBatchNorm2d()
                  (drop_path): Identity()
                )
              )
            )
          )
        )
        (1): PositionEmbeddingSine()
      )
      (answer_type_head): Linear(in_features=256, out_features=5, bias=True)
      (answer_rel_head): Linear(in_features=256, out_features=1594, bias=True)
      (answer_obj_head): Linear(in_features=256, out_features=3, bias=True)
      (answer_global_head): Linear(in_features=256, out_features=111, bias=True)
      (answer_attr_head): Linear(in_features=256, out_features=403, bias=True)
      (answer_cat_head): Linear(in_features=256, out_features=678, bias=True)
    )




```python
def plot_inference_qa(im, caption):
    img = transform(im).unsqueeze(0).cuda()

    # モデル伝搬
    memory_cache = model_qa(img, [caption], encode_and_save=True)
    outputs = model_qa(img, [caption], encode_and_save=False, memory_cache=memory_cache)

    probas = 1 - outputs['pred_logits'].softmax(-1)[0, :, -1].cpu()
    keep = (probas > 0.7).cpu()

    bboxes_scaled = rescale_bboxes(outputs['pred_boxes'].cpu()[0, keep], im.size)

    # 各ボックスで予測されるテキストスパンを抽出
    positive_tokens = (outputs["pred_logits"].cpu()[0, keep].softmax(-1) > 0.1).nonzero().tolist()
    predicted_spans = defaultdict(str)
    for tok in positive_tokens:
        item, pos = tok
        if pos < 255:
            # Encoderで処理したテキスト情報から、テキストスパンを取得
            span = memory_cache["tokenized"].token_to_chars(0, pos)
            # 入力文字(caption)から対応する文字を取得
            predicted_spans [item] += " " + caption[span.start:span.end]

    labels = [predicted_spans [k] for k in sorted(list(predicted_spans .keys()))]
    plot_results(im, probas[keep], bboxes_scaled, labels)

    # 質問タイプの分類
    type_conf, type_pred = outputs["pred_answer_type"].softmax(-1).max(-1)
    ans_type = type_pred.item()
    types = ["obj", "attr", "rel", "global", "cat"]

    # 質問タイプから一番マッチする回答をIDからマッピングして取得
    ans_conf, ans = outputs[f"pred_answer_{types[ans_type]}"][0].softmax(-1).max(-1)
    answer = id2answerbytype[f"answer_{types[ans_type]}"][ans.item()]
    print(f"Predicted answer: {answer}\t confidence={round(100 * type_conf.item() * ans_conf.item(), 2)}")
```

下記で推論を実行


```python
url = "https://s3.us-east-1.amazonaws.com/images.cocodataset.org/val2017/000000076547.jpg"
im3 = Image.open(requests.get(url, stream=True).raw)
plot_inference_qa(im3, "What is on the table?")
```


    
![output_42_0.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/606924/ca386726-50df-45de-b1c2-46c3de761d87.png)
    


    Predicted answer: no	 confidence=nan
    

推論した結果、「answer」は「no」、「confidence」はnan値となり、処理は正しく行われない。

入力データの「img」、出力の「memory_cache」と「outputs」の中身を見ると、


```python
img = transform(im3).unsqueeze(0).cuda()
caption = "What is on the table?"

# モデル伝搬
memory_cache = model_qa(img, [caption], encode_and_save=True)
outputs = model_qa(img, [caption], encode_and_save=False, memory_cache=memory_cache)
```


```python
img
```




    tensor([[[[-1.7754, -1.7412, -1.6727,  ...,  0.3309,  0.1939,  0.0912],
              [-1.7412, -1.7069, -1.6727,  ...,  0.4851,  0.3823,  0.3138],
              [-1.6727, -1.6727, -1.6727,  ...,  0.7248,  0.6734,  0.6392],
              ...,
              [-1.3302, -1.3987, -1.5014,  ..., -0.7993, -0.7822, -0.7650],
              [-1.4500, -1.4672, -1.5014,  ..., -0.7822, -0.7993, -0.7993],
              [-1.5185, -1.5185, -1.5014,  ..., -0.7822, -0.7993, -0.8164]],
    
             [[-1.6856, -1.6331, -1.5455,  ...,  0.4853,  0.4153,  0.3627],
              [-1.6681, -1.6331, -1.5630,  ...,  0.6604,  0.6254,  0.6078],
              [-1.6506, -1.6155, -1.5805,  ...,  0.9230,  0.9580,  0.9930],
              ...,
              [-1.1954, -1.2304, -1.3004,  ..., -0.3200, -0.2850, -0.2500],
              [-1.3004, -1.3004, -1.2829,  ..., -0.3025, -0.3200, -0.3200],
              [-1.3704, -1.3354, -1.2829,  ..., -0.3025, -0.3375, -0.3725]],
    
             [[-1.4559, -1.4036, -1.3339,  ...,  0.7751,  0.7751,  0.7751],
              [-1.4384, -1.4036, -1.3339,  ...,  0.9668,  0.9494,  0.9145],
              [-1.4036, -1.3861, -1.3513,  ...,  1.2631,  1.1934,  1.1411],
              ...,
              [-0.7587, -0.7936, -0.8633,  ..., -0.6715, -0.5844, -0.5321],
              [-0.9156, -0.8981, -0.8807,  ..., -0.6367, -0.6367, -0.6367],
              [-1.0201, -0.9678, -0.8807,  ..., -0.6193, -0.6715, -0.7064]]]],
           device='cuda:0')




```python
memory_cache
```




    {'text_memory_resized': tensor([[[ 0.0463,  0.0675,  0.1477,  ...,  0.0164,  0.2151,  0.0931]],
     
             [[-0.5830,  0.0047,  0.8694,  ..., -0.5799, -0.6114, -0.4202]],
     
             [[-0.5820, -0.0295,  0.8522,  ..., -0.6351, -0.6301, -0.4519]],
     
             ...,
     
             [[-0.7565,  0.5123, -0.3289,  ..., -0.4070,  0.3803, -0.2708]],
     
             [[ 0.0463,  0.0675,  0.1477,  ...,  0.0164,  0.2151,  0.0931]],
     
             [[ 0.0463,  0.0675,  0.1477,  ...,  0.0164,  0.2151,  0.0931]]],
            device='cuda:0'),
     'text_memory': tensor([[[nan, nan, nan,  ..., nan, nan, nan]],
     
             [[nan, nan, nan,  ..., nan, nan, nan]],
     
             [[nan, nan, nan,  ..., nan, nan, nan]],
     
             ...,
     
             [[nan, nan, nan,  ..., nan, nan, nan]],
     
             [[nan, nan, nan,  ..., nan, nan, nan]],
     
             [[nan, nan, nan,  ..., nan, nan, nan]]], device='cuda:0'),
     'img_memory': tensor([[[nan, nan, nan,  ..., nan, nan, nan]],
     
             [[nan, nan, nan,  ..., nan, nan, nan]],
     
             [[nan, nan, nan,  ..., nan, nan, nan]],
     
             ...,
     
             [[nan, nan, nan,  ..., nan, nan, nan]],
     
             [[nan, nan, nan,  ..., nan, nan, nan]],
     
             [[nan, nan, nan,  ..., nan, nan, nan]]], device='cuda:0'),
     'text_pooled_op': None,
     'img_pooled_op': None,
     'mask': tensor([[False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False, False, False,
              False, False, False, False, False, False, False, False]],
            device='cuda:0'),
     'text_attention_mask': tensor([[False, False, False, False, False, False, False, False]],
            device='cuda:0'),
     'pos_embed': tensor([[[2.4869e-01, 9.6858e-01, 2.1593e-01,  ..., 1.0000e+00,
               2.1340e-05, 1.0000e+00]],
     
             [[2.4869e-01, 9.6858e-01, 2.1593e-01,  ..., 1.0000e+00,
               4.2681e-05, 1.0000e+00]],
     
             [[2.4869e-01, 9.6858e-01, 2.1593e-01,  ..., 1.0000e+00,
               6.4021e-05, 1.0000e+00]],
     
             ...,
     
             [[0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
               0.0000e+00, 0.0000e+00]],
     
             [[0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
               0.0000e+00, 0.0000e+00]],
     
             [[0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
               0.0000e+00, 0.0000e+00]]], device='cuda:0'),
     'query_embed': tensor([[[ 0.0491,  1.1184, -0.2196,  ...,  0.4614, -0.0693,  0.6637]],
     
             [[-0.6772, -0.7437, -0.5867,  ...,  0.1200,  1.8001,  0.5810]],
     
             [[ 1.9782,  0.2545, -0.6935,  ...,  0.1258, -1.6969, -0.6355]],
     
             ...,
     
             [[ 0.4497,  0.5344,  0.3115,  ..., -0.6557,  0.1694, -0.1874]],
     
             [[-2.3338, -0.9938, -0.3027,  ...,  0.2182,  0.9463,  0.0895]],
     
             [[-1.5761, -0.5586, -0.4865,  ..., -1.8347, -1.2846,  0.0172]]],
            device='cuda:0'),
     'tokenized': {'input_ids': tensor([[   0, 2264,   16,   15,    5, 2103,  116,    2]], device='cuda:0'), 'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1]], device='cuda:0')}}




```python
outputs
```




    {'pred_answer_type': tensor([[nan, nan, nan, nan, nan]], device='cuda:0'),
     'pred_answer_obj': tensor([[nan, nan, nan]], device='cuda:0'),
     'pred_answer_rel': tensor([[nan, nan, nan,  ..., nan, nan, nan]], device='cuda:0'),
     'pred_answer_attr': tensor([[nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan]],
            device='cuda:0'),
     'pred_answer_cat': tensor([[nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan]], device='cuda:0'),
     'pred_answer_global': tensor([[nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
              nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan]],
            device='cuda:0'),
     'pred_logits': tensor([[[nan, nan, nan,  ..., nan, nan, nan],
              [nan, nan, nan,  ..., nan, nan, nan],
              [nan, nan, nan,  ..., nan, nan, nan],
              ...,
              [nan, nan, nan,  ..., nan, nan, nan],
              [nan, nan, nan,  ..., nan, nan, nan],
              [nan, nan, nan,  ..., nan, nan, nan]]], device='cuda:0'),
     'pred_boxes': tensor([[[nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan],
              [nan, nan, nan, nan]]], device='cuda:0')}



nan値がかなり多い結果となっています。
「memory_cache」の時点で、既にnan値が入っているので、画像・言語から中間ベクトルを算出する始めのEncoderの所で、うまく動作していない模様。

# 推論がうまく動作しない

諦める。