Here is my experimental design:
- Total of 1028 reference images, evenly divided into 4 categories: animals, people, nature, and food (256 images per category).
- Each image can have 3 distortions applied to them (01, 02, 03), and each distortion has 3 levels (A, B, C).

- I want approximately 300 observers to rate around 400 reference images each such that on the observer level:
1. Each observer rates an equal number of images from each of the 4 categories.
2. The distortions and levels are evenly applied within each category.
3. Each reference image is maximally seen once by each observer.

On a group level, all reference images should receive a similar number of ratings if possible.

There have been several issues all of which should be avoided. Each observer could see an unbalanced number of categories, e.g. only animals. There could be way to few images per observer e.g. 35. Which the following code outputs:

import csv
import random
from itertools import cycle

# Create a list of image categories and their corresponding range of image indices
categories = {
    'animals': range(1, 257),
    'people': range(257, 513),
    'nature': range(513, 769),
    'food': range(769, 1025)
}

# Define the distortions and levels, and adding an 'original' distortion type
distortions = ['01', '02', '03', 'orig']
levels = ['A', 'B', 'C']

# Function to create a list of image combinations for a given category, distortion, and level
def image_combinations(category, distortion, level):
    image_indices = categories[category]
    if distortion == 'orig':
        images = [(f"{idx}orig{category}",) for idx in image_indices]
    else:
        images = [(f"{idx}{distortion}{level}_{category}",) for idx in image_indices]
    return images

all_image_combinations = []

# Create a list of all possible image combinations
for category in categories:
    cat_combinations = []
    for distortion in distortions:
        if distortion == 'orig':
            cat_combinations.extend(image_combinations(category, distortion, None))
        else:
            for level in levels:
                cat_combinations.extend(image_combinations(category, distortion, level))
    all_image_combinations.append(cat_combinations)

num_observers = 300
images_per_observer = 416
observer_images = [[] for _ in range(num_observers)]

# Distribute images among observers
range_cycle = cycle(range(num_observers))  # Cycle over the range of observers infinitely
for i, cat_images in enumerate(all_image_combinations):  # Iterate over categories
    random.shuffle(cat_images)  # Shuffle before assigning images
    # Set a quota for number of images for each observer within the category
    images_per_observer_count = [0] * num_observers
    # Iterate over each image in the category combinations
    for img_idx, img in enumerate(cat_images):
        # Select an observer based on img_idx
        observer_idx = next(range_cycle) % num_observers
        # Attempt to assign the image to an observer if that observer hasn’t reached the quota within the category
        max_trial = 100
        trial = 0
        while (images_per_observer_count[observer_idx] >= images_per_observer // len(categories) and trial < max_trial):
            observer_idx = next(range_cycle) % num_observers
            trial += 1
        observer_images[observer_idx].append(img)
        images_per_observer_count[observer_idx] += 1

# Shuffle assigned images within each observer
for observer in observer_images:
    random.shuffle(observer)

# Write the observer images to separate CSV files
for observer_id, images in enumerate(observer_images, 1):
    filename = f"conditions{observer_id}.csv"
    with open(filename, 'w', newline='') as csvfile:
        writer = csv.writer(csvfile)
        writer.writerow(["image"])
        for img, in images:
            writer.writerow([f"{img}.jpg"])