我正在實作implement a conditional Denoising Diffusion Probabilistic Model (DDPM) to generate synthetic images according to multi-label conditions: Given a specific set of label conditions, your DDPM should generate the corresponding synthetic images (see Fig. 1). For example, “red sphere,” “yellow cube” and “gray cylinder” are given, your model should generate the synthetic images containing those objects. Meanwhile, you also need to input your generated images into a pre-trained classifier for evaluation.

Requirements
Prepare dataset ● Implement data loader ● Define your multi-label condition embedding method
Implement a conditional DDPM ● Choose your conditional DDPM setting, design your model architecture ● Design your noise schedule, time embeddings, sampling method, etc. You can also include the pretrained evaluator as a “classifier guidance” in sampling ● Choose your loss functions (e.g. reparameterization type) ● Implement the training function, testing function - Evaluate the results ● Evaluate the accuracy of test.json and new test.json, input the output images of each model to the pretrained evaluator to get the accuracies ● Show the synthetic images in grids for two models * two testing files ● Show the denoising process in a grid for sampling an image from your DDPM, with the label set ["red sphere", "cyan cylinder", "cyan cube"]

Basic Rules ● For each model, you can choose any architecture you like. Also, feel free to use any library that helps your work, but you need to mention the implementation details and include the reference in your report. ● Except for the provided files, you cannot use other training data. (e.g. background image) ● Use the function of a pre-trained evaluator to compute the accuracy of your synthetic images. ● The evaluator is based on ResNet18. By inputting synthesized images and one-hot GT labels, the evaluator will return accuracy. Its class script and pretrained weight are given. DO NOT modify any of them, BUT involving the classification result in your discriminator or classifier guidance is allowed. Please inherit the class if you need extra functionalities. ● The normalization of input images is transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)). You must input the normalized image into the pretrained classifier. You should apply denormalization to generate RGB images. ● The resolution of input for the pretrained evaluator is 64x64. You can design your own output resolution and resize it for evaluation ● Use make_grid(images) from torchvision.utils to generate grids for your testing image (8 images a row, 4 rows for each JSON file) and denoising process (at least 8 images in 1 row from noisy to clear). ● If you have no idea how to design your DDPM, follow 3.2 for simple suggestions.

Scoring Criteria

Report (60%) ● Introduction (5%) ● Implementation details (25%) o Describe how you implement your model, what is this step, including your choice of DDPM, noise schedule. (Please write in detail.) ● Results and discussion (30%) o Show your synthetic image grids (total 16%: 8% * 2 testing data) and a denoising process image with the label set ["red sphere", "cyan cylinder", "cyan cube"] (4%) o Discussion of your extra implementations or experiments (10%)
Experimental results (40%) (based on results shown in your report) ● Classification accuracy on test.json and new test.json (40%) ● Show your accuracy screenshots (20% * 2 testing data)
This is the specification file for conditional DDPM training and testing dataset - i-CLEVR

objects.json This file is a dictionary file that contains the number of objects and the idexes. There are totally 24 objects in i-CLEVR datasets with 3 shapes and 8 colors.

train.json The file is for training. The number off training data is 18009. train.json is a dictionary where keys are filenames and values are objects/ For example: {"CLEVR_train_001032_0.png": ["yellow sphere"], "CLEVR_train_001032_1.png": ["yellow sphere", "gray cylinder"], "CLEVR_train_001032_2.png": ["yellow sphere", "gray cylinder", "purple cube"], ... } One image can include objects from 1 to 3

test.json, new_test.json The file are for testing. The number of testing data is 32 for each file. Both file contain a list where each element includes multiple objects For example: [['gray cube'], ['red cube'], ['blue cube'], ['blue cube', 'green cube'], ...]

evlauator.py, checkpoint.pth Here are the pretrained classifier model's script and its weight for evaluating your synthetic images. More detail can be found in the script file.

'''===============================================================

Title:
DLP Spring 2025 Lab6 classifier

Purpose:
For computing the classification accruacy.

Details:
The model is based on ResNet18 with only chaning the last linear layer. The model is trained on iclevr dataset with 1 to 5 objects and the resolution is the upsampled 64x64 images from 32x32 images.

It will capture the top k highest accuracy indexes on generated images and compare them with ground truth labels.

How to use
You may need to modify the checkpoint's path at line 40. You should call eval(images, labels) and to get total accuracy. images shape: (batch_size, 3, 64, 64) labels shape: (batch_size, 24) where labels are one-hot vectors e.g. [[1,1,0,...,0],[0,1,1,0,...],...] Images should be normalized with: transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))

我現在要開始寫report，協助我完成