How to generate the Grouding DINO model architecture graph with tensorboard?

username: Young_Andy
cooked: <p>These days, a popular open-set object detector called Grounding DINO was proposed.  By marrying Transformer-based detector DINO with grounded pre-training, which can detect arbitrary objects with human inputs such as category names or referring expressions.  I try to visualize the model architecture with tensorboard. But when I pass the image and ‘TEXT_PROMPT’ after the preprocess_caption function, the error comes "RuntimeError: Type ‘Tuple[Tensor, List[str]]’ cannot be traced. The code block is as follows:</p>
<pre><code class="lang-auto">import os

def preprocess_caption(caption: str) -&gt; str:
result = caption.lower().strip()
if result.endswith("."):
return result
return result + "."

CONFIG_PATH = os.path.join(HOME, "groundingdino/config/GroundingDINO_SwinT_OGC.py")
print(CONFIG_PATH, "; exist:", os.path.isfile(CONFIG_PATH))

WEIGHTS_NAME = "groundingdino_swint_ogc.pth"
WEIGHTS_PATH = os.path.join(HOME, "weights", WEIGHTS_NAME)
print(WEIGHTS_PATH, "; exist:", os.path.isfile(WEIGHTS_PATH))

IMAGE_NAME = "dog-3.jpeg"
IMAGE_PATH = os.path.join(HOME, "data", IMAGE_NAME)

TEXT_PROMPT = "chair"
TEXT_PROMPT = preprocess_caption(TEXT_PROMPT)
BOX_TRESHOLD = 0.35
TEXT_TRESHOLD = 0.25

model = load_model(CONFIG_PATH, WEIGHTS_PATH)

image_source, image = load_image(IMAGE_PATH)
with torch.no_grad():
outputs = model(image[None], captions=[TEXT_PROMPT])

log_dir = "./logs"
writer = SummaryWriter(log_dir)

writer.add_graph(model, input_to_model={"samples": image[None], "captions": TEXT_PROMPT})
writer.add_graph(model,(image[None], [TEXT_PROMPT]))
writer.close()
</code></pre>
<p>the error comes “Only Tensors and (possibly nested) Lists, Dicts, and Tuples of Tensors can be traced”.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/b/1bcfa9ab0945d35d8ab47c2287365d6135743301.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/1bcfa9ab0945d35d8ab47c2287365d6135743301" title="GRDINO_visual_issue"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/b/1bcfa9ab0945d35d8ab47c2287365d6135743301_2_690x277.jpeg" alt="GRDINO_visual_issue" data-base62-sha1="3Y1O2APVnCNr67f3V9i271mGVPj" width="690" height="277" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/b/1bcfa9ab0945d35d8ab47c2287365d6135743301_2_690x277.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/1/b/1bcfa9ab0945d35d8ab47c2287365d6135743301_2_1035x415.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/1/b/1bcfa9ab0945d35d8ab47c2287365d6135743301_2_1380x554.jpeg 2x" data-dominant-color="222323"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">GRDINO_visual_issue</span><span class="informations">2993×1202 309 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
Because the text and image backbone deal with corresponding data at the beginning of Grounding DINO, How can I pass the input to the add_graph function? I also tried the netron.app with the .pth file, but it didn’t have the link between the layers.<br>
Could you please give some advice to solve the visualization problem? Thanks!</p>