SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 7: Co attention mechanisms (P-SA, G-SA) and the door matching loss  $(L_{\mathrm{match}})$ . Full RPLAN is used.  $\checkmark$ indicates the feature being used. In case of  $L_{\mathrm{match}}$  "Doors" means matching loss has been applied only on door corners and "All corners" means matching loss has been applied to all corners including door corners.
P-SA	G-SA	Lmatch	MPE (↓)	GED (↓)
✓	25.6	1.6
✓	✓	24.2	1.5
✓	Doors	36.9	2.4
✓	Doors	22.1	1.1
✓	✓	All Corners	10.7	0.9
✓	✓	Doors	10.5	0.9
Table 8: Effects of the room-type (R-type) and the Door information. Full RPLAN is used.  $\checkmark$  indicates the information being used. When a room-type is not used, we set a zero vector as a room-type one-hot vector. When the door information is not used, we do not pass the door-corner nodes to the network.
Train	Test	MPE(↓)	GED (↓)
R-Type	Door	R-Type	Door
✓	✓	✓	17.3	1.4
✓	✓	16.4	1.5
✓	✓	✓	15.1	1.9
✓	✓	14.3	1.9
✓	✓	✓	✓	10.5	0.9
Predicted Arrangement Ground Truth
Predicted Arrangement Ground Truth
Figure 11: Qualitative evaluations of our method for Full MagicPlan dataset without GT rotations top two rows, and Full RPLAN dataset without GT rotations bottom two rows. We show edges and corners here to show overlaps and noisy annotations more clear.
C.2 Additional ablation studies on puzzle solving
We have provided additional qualitative results of our method in Figure 15 and Figure 16 including noisy samples or samples with missing or duplicate pieces. In case of missing and duplicate experiment, we repeat (remove) each piece with a probability of  $10\%$ . Table 9 presents the evaluation metrics of missing and duplicate experiments.
C.2.1 Pictorial Cross-cut Jigsaw Puzzle
To enhance the integration of image information into our pictorial puzzle diffusion models, we employed a two-step approach. Firstly, we pretrained an auto-encoder utilizing the puzzle pieces. This auto-encoder served as the image embedder for our diffusion model, enabling the conversion of each puzzle into a compact 128D feature vector.
The pretraining process involved training the model to downsample an input image of dimensions  $3 \times 256 \times 256$  to a compressed representation of size  $32 \times 2 \times 2$  within the encoder, and subsequently reconstructing the original image size in the decoder. We employed the mean squared error
Figure 12: RLA arrangement results with Figure 13: The final room arrangement before and after Full MagicPlan dataset. On left two successive averaging. Our diffusion model estimates a room-cessful cases, on right two failed cases.. Our position/rotation at each room corner, which may not failures are often attributed to 1) Rare build- be consistent in a room. The final arrangement is ing architecture (top-right) and 2) Inherent obtained by taking the average position and rotation ambiguity (bottom-right), whose tasks are within each room.
Figure 14: A diffusion model is stochastic and produces a different result every time. The middle rows show five different pose estimation results. The top (resp. bottom) is from Full RPLAN (resp. Full MagicPlan) dataset.
(MSE) loss function during training. However, to focus our model's attention on learning the texture features, given that the diffusion model already captured the geometry features, we applied the loss function exclusively to the pixels within the puzzle piece.
By adopting this selective application of the loss function, we prioritize the acquisition of texture-based details, as the geometric characteristics are already embedded within the diffusion model.
Quantitatively, we also evaluated our method on the full Cross-cut dataset to measure the effectiveness of the pictorial information compared to apictorial scenario. We found that the model converges faster when using pictorial information while it achieves slightly better overlap score of 0.9417 compared to 0.9398 in apictorial scenario. Figure 17 shows additional qualitative results of our method compared to Harel et al. Harel and Ben-Shahar (2021).
Table 9: Effects of the Missing or Duplicate pieces in puzzle solving problem.  $\checkmark$  indicates it if missing or duplicate piece were presented during test time. In training time we do not have duplicate or missing piece presented to show our model robustness to unseen noise during test.
Cross-cut	Voronoi
Missing	Duplicate	Overlap (↑)	Precision (↑)	Recall (↑)	Overlap (↑)	Precision (↑)
✓	-	0.88	0.92	0.82	0.68	0.68
-	✓	0.92	0.97	0.88	0.67	0.71
-	-	0.94	0.97	0.91	0.70	0.78
Figure 15: Additional qualitative results of Voronoi jigsaw puzzle are presented in four different setups: 1) No noise, 2) Noise level 2, 3) Missing piece, and 4) Duplicate piece (D indicates the duplicated pieces).
Figure 16: Additional qualitative results of Cross-cut jigsaw puzzle are presented in four different setups: 1) No noise, 2) Noise level 2, 3) Missing piece, and 4) Duplicate piece (D indicates the duplicated pieces).
Harel et al.
Figure 17: Additional qualitative results of pictorial Cross-cut jigsaw puzzle compared to Harel et al. Harel and Ben-Shahar (2021).