In diffusion models, the learning dynamics and representation formation play a crucial role in their ability to compositionally generalize. The study conducted on a toy model investigating the generation of 2D Gaussian bumps at varying positions revealed three distinct learning phases: no latent structure, a disordered 2D manifold, and an ordered 2D manifold. These phases corresponded to different generation behaviors and failure modes. Initially, without a specific structure, the model struggled to generate accurate Gaussian bumps. As training progressed, the model transitioned to a quasi-2D unordered manifold before finally reaching an ordered 2D manifold, indicative of successful learning and generation of the desired Gaussian bumps. This progression highlights the importance of the internal representation learned by the model in achieving compositional generalization. A semantically meaningful and fully factorized representation is essential for the model to accurately generate complex compositions, such as the unconventional images seen in real-world datasets.