SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Additionally, we investigated the quality of learned graphs and the latent parameters, and their effect on the RL performance w.r.t. the number of samples for the 3-Push + 3-Switch (L+O+S) task. In particular, we varied the amount of training data, ranging from  $10\%$ ,  $20\%$ ,  $40\%$ ,  $60\%$ ,  $80\%$  of the original sample size (900 trajectories). We measured the  $R^2$  coefficient of our learned parameters with the true latent parameters, both for a random policy (Fig. A4a) and for a pre-trained policy (Fig. A4b). These results show that the performance degrades with smaller sample sizes, but it is still acceptable with  $60\%$  data, and that the difference between data collected with a random or pre-trained policy is negligible at higher sample sizes; the normalized Structural Hamming Distance between the reconstructed graph and the ground truth graph, as well as the success rate of the learned policy. As expected, the more data, the more accurate the graph, and the better the performance of the policy trained with the more accurate world model. For the number of samples in the main paper, the graph is perfectly reconstructed, which also means a good RL performance. Additionally, for limited data (e.g. 0.2 or 0.4 of the original dataset) leveraging a pre-trained policy enhances both graph and policy learning. However, as the amount of data increases, the benefits of using a pre-trained policy diminish.
Table A10: Comparisons on the using AIR or SA as the object-centric model for symbolic methods.
Experiment Settings	Method
DAFT-RL (SA)	DAFT-RL (AIR)	SMORL	SRICS	GNN	STOVE	NCS	LRN
Original	0.506 ±0.083	0.571 ±0.039	0.386 ±0.062	0.420 ±0.061	0.334 ±0.047	0.278 ±0.086	0.397 ±0.052	0.463 ±0.077
True latent parameter	0.525 ±0.062	0.598 ±0.023	0.475 ±0.081	0.501 ±0.072	0.386 ±0.035	0.315 ±0.049	0.410 ±0.075	0.492 ±0.068
Parameter learned by DAFT-RL	0.506 ±0.083	0.571 ±0.039	0.415 ±0.059	0.493 ±0.073	0.359 ±0.041	0.263 ±0.085	0.399 ±0.049	0.484 ±0.061
Table A11: Effects of the latent parameter modeling. Original: Results in the original experiments, where for DAFT-RL, we use the learned latent and the others are without latent parameter estimations; True latent parameter: All methods directly use the true latent as part of the states. Parameters learned by DAFT-RL: all baselines are using the latent learned by DAFT-RL as part of the state vectors. Average success rate over 3 random seeds for Block-stacking with 8 boxes and different masses. The numbers in bold highlight the top-performing method.
Figure A4: Quality of learned graphs w.r.t. the number of samples for 3-Push + 3-Switch (L+O+S). We plot the success rate of the RL task, the  $R^2$  coefficient for learned representation vs the true latent parameters, and normalized Structured Hamming Distance (nSHD) between the learned graph and true graph under different number of training samples (measured as a ratio with the ones in the main paper) for (a) data collected by random policy, and (b) data collected by pre-trained policy.
D.7.1 Visualization of the Learned Graphs
In Figure A5, which illustrates the learned graphs for boxes and switches in the Push & Switch benchmarks, we denote the class template graphs for boxes and switches as A1 and A2. The variables
$C_1.s_1$ ,  $C_1.s_2$ ,  $C_1.s_3$ , and  $C_1.s_4$  represent the color, shape, velocity, and position of the boxes, respectively. Similarly,  $C_2.s_1$ ,  $C_2.s_2$ ,  $C_2.s_3$ , and  $C_2.s_4$  correspond to the color, shape, position, and angle of the switches. In both cases,  $C_i.\theta$  and  $C_j.\theta$  indicate the friction coefficients.
As expected, in the learned class template graph, we observe that shape and color do not have a direct effect on the dynamics, and they remain unchanged throughout the dynamics. On the other hand, velocity directly influences the position of the boxes, while the position itself does not affect the velocity. Regarding the switches, their positions do not directly impact the angles since they are fixed at specific positions on the panel. Also, as expected, actions applied to the switches have an effect solely on their angles. The reward is based on the actions, positions, and angles for both boxes and switches. Additionally, the latent parameters influence the velocity and position of the boxes, while they specifically affect the angle of the switches.
B
Figure A5: Learned class template graph and interaction pattern graph in push & switch task. A1-A2: Class template graphs for box and switch. B: interaction pattern graph among two boxes.
As for the interaction graph between two boxes, shown in Figure A5B, the velocity of one box can impact the position of the other box, while the other variables do not have a direct influence on each other. This learned interaction graph successfully captures the underlying physical processes of the system, enabling an attribute-level understanding of the object interactions. Overall, these learned graphs effectively recover the underlying physical processes of the system.