A fascinating approach to deep generative design
By: Dickson Wu
Tags: Design, UX, Generative Design, Artificial Intelligence, Machine Learning

Collect previous design data. This is used later in stages 2 and 7. And it’s also benchmark data to compare the newly generated designs. Designs enter here, and new designs are spit right out. This is done through topology optimization — which is a multi-objective function: 1) Maintain engineering performance. 2) L1 loss of the reference. Different designs are obtained by weighing the objectives in different ways (hyperparameter). Now we have a ton of designs — but we don’t want to waste computation on irrelevant (aka designs that are too similar). So we filter similar designs! So we do a pixel-wise L1 loss. The user defines the threshold (hyperparameter), the higher it is, the tighter the constraint, the fewer designs that pass, but the more diverse they are. An important note is that if the generator rotated the image, L1 won’t be able to pick it up and call them different. We can use generative models to map designs to a latent space, but this paper didn’t do it because rotations didn’t occur too often If new designs / total designs from previous iteration < user-defined threshold — then you exit the loop and go to stage 8. Else you continue on to stage 5. A GAN (more specifically a Boundary Equilibrium GAN (BEGAN)) is used to generate new designs. (it will be explained in detail later in the section about the generative models) Just like stage 3, which filters out similar designs. Outputs of stage 6 are inputted back into stage 2. This is where we build the loss function — which is an autoencoder that’s trained on stage 1 designs. The loss function is to evaluate the novelty of designs (details are also in the later section about generative models) We use the loss function, along with other evaluation metrics to get evaluations for each design — things like novelty, volume, compliance to designs. Finally, we use those data points and visualize them on a graph. We can then choose designs based on this graph.

Topology Optimization:

A High-Level Overview:

Credit: Jun Wu’s Lectures

On a high level, Topology Optimization is just iterating over the design over and over again until we reach an optimal design. Let’s just say we’re building a bridge. We first get the model to just guess where we should place the material (x_e).

Then we compute the displacement (like getting the loss in ML) to see how good our design is. Afterwards, we do sensitivity analysis (like getting the gradient for each weight in NNs), with respect to the displacement.

This sensitivity analysis will tell us how important each part of the design is. The parts which are important have additional material added to them. The unimportant parts will have the material taken away from it

We do this over and over again until the design has converged! Now we have an optimized design that can withstand the engineering requirements with as little material as possible!

The Nitty Gritties:

Topology Optimization is all about trying to find the optimum design for a specific task. In most cases, you want to maximize stiffness — otherwise known as minimizing the elastic energy of the system:

Where U is the displacement vector

K is the global stiffness matrix

c(x) is the compliance

u_e is the element displacement vector

k_0 is the element stiffness matrix

N_e is the total number of elements

x_e is the design variable (density)

Why does this look a bit familiar? Well, it’s because of Hooke’s law:

Cool! Do we just let it run now? Nope — we have some constraints we have to place on the design: like Volume

V(x) is the volume that the design takes up

V_0 is the volume of the entire design domain

f is a user-defined fraction of the volume that the design is allowed to take up

So the model starts off with this play area in which it can fill. When it starts filling the area, it won’t be filling it in as 0 and 1 → It’s too difficult for the program to learn like that. Thus we allow it to learn a continuous plane:

But we’re using Solid Isotropic Material with Penalization (SIMP) to encourage the model to go towards 0 and 1 such that we do get a clean black and white (0 and 1) image. Which looks like this:

p is the penalization factor

E_min is to avoid instability when the element becomes a 0

Great! Now it has an objective with some constraints. But how does it iterate and create better designs? It’s time for sensitivity analysis! Just like in machine learning where we find the partial derivative of loss with respect to the parameter, we’re doing the same here:

All elements have a unit volume:

And we update the material (x_e) by:

And the termination condition (when it converges) is:

Ɛ is a user set value — usually something like 0.01

Are we done? Nope! This usually gives an ugly checkerboard pattern + other artifacts, thus this paper uses a 3 field SIMP projection scheme to avoid it. Here we apply density and sensitivity filters on the neighbourhood of points (like a convolution).

Sensitivity filter:

Density filter:

Now if we just used those two it still wouldn’t be enough because there are grey transition pixels — thus we have a projection filter:

Now we’re done with Topology Theory! Now let’s move on to what this study did:

What This Study Did:

The entire design space is 128 x 128 pixels → But we won't be letting it just put stuff wherever it wants. Instead, we’ll set some areas which it isn’t allowed to place:

Thus it’s only the light grey part in which generative design is actually allowed to play with (the spoke). The force we’ll be putting on it is the normal force + the sheer force tire pressure + traction forces — and we can change the ratio between them.

To make sure that our generate creates designs that match reference designs, we create an objective function that’s equal to the compliance (how stiff the wheel is against external forces) plus the L1 Norm between inputs and outputs:

First part = the compliance

Second part = the L1 loss part (with a ratio of λ to specify which one we want to weigh more)

And our sensitivity analysis looks like this:

There are different parameters with which we can play with. In this study, there are 2 sets of parameters we can adjust, each with 5 discrete levels.

And thus we’re able to run our framework on 25 different parameter combinations!

Generative Models:

The true data has a distribution: p_data(x) and the goal of the generator is to create data to match this distribution. The generator gets inputted with noise, and then outputs data in distribution p_data(x).

Meanwhile, this is going on, we have a discriminator that tries to figure out which images are from the original p_data(x) distribution, and which are from the generator. It’s a zero-sum game between the discriminator and the generator → Both of their loss functions are dependent on the other’s performance (the lower the performance of the opponent, the lower your loss is!).

There are tons of sub-branches of GANs — We have DCGANs (Deep convolutional GANs, ALI (Adversarially Learned Inference), BiGANs (Bidirectional GANs), EBGANs (energy-based GANs), WGANs (Wasserstein GANs), InfoGANs (Information maximizing GANs), cGANs (Conditional GANs) and BEGANs (Boundary Equilibrium GANs).

This paper uses BEGANs. BEGANs use an equilibrium concept, which balances the discriminator and generator to global convergence. It also uses the autoencoder architecture, and the Wasserstein distance → Thus enabling high-resolution images to be generated.

BEGANs also match the autoencoder loss distribution rather than the probability distribution — because the generator is actually the decoder for the autoencoder:

In a similar vein, we’re able to detect novelty using autoencoders → Since we’re able to effectively reconstruct designs that we’ve seen during training, but dissimilar data won’t be constructed as well. Thus we’re using autoencoders for stage 7!

Results:

Different types of parameters end up with different types of designs in topologic optimization:

Parameter for force:

Parameter for similarity:

We can also see that different reference designs end up with different optimized designs (meaning we have lots of diversity)

Now it’s time for the GANs:

What’s amazing is that the outputs look pretty feasible — they’re all symmetric, have holes and have around the same boundary for the rim.

Let’s test out the novelty evaluation:

We can see that it’s really bad at reconstructing designs that it’s never seen before — affirming that the autoencoder works for novelty detection!

And just after 1.5 epochs (since the second epoch triggered the termination criteria) we got 2004 new designs:

Here are some examples of generated designs:

And they plotted the wheels and their factors:

From these 2D designs, we can then turn them into 3D designs!

Discussion:

Topology Optimization:

They first started off by seeing: are references actually necessary? So they took out references and only got it to loop through topology optimization → Results had identical outcomes if the parameters were the same, thus there was no aesthetic diversity. Sometimes it also failed to converge. Therefore references augmented it.

Additionally, they validate the use of GAN’s (since they morph the fundamental topology of the wheel). Because if you only use topologic optimization, similar reference designs will, after iterations, have results that seem the same as well:

BEGAN:

Now we move on to the GAN. One thing to note: the genius of the framework proposed in this paper is that the GAN doesn’t have to be perfect — since topology optimization can take over! It’s quite robust!

They tested it out with other GANs like DCGAN and VAE. But they were blurrier, less symmetrical and lacked novelty:

BEGAN is a good choice becomes it creates topologically novel designs that are simple — which is a key to better inputs for better designs for topologic optimization.

Future work lies in training a model which is able to select the good models from multiple generative networks, along with different latent variables.

Autoencoder:

Afterwards, they tested the effectiveness of the autoencoder — they did so by seeing how accurate the model was at predicting between reference designs and generated models. It did so with a 91.6% accuracy:

Conclusion:

This paper puts forth an amazing framework! It combines the topological optimation along with GANs in a cyclical and iterative fashion in order to achieve novel aesthetic and engineering performance in generated designs simultaneously. They can start off with a tiny amount of designs and explode them to create tons of new designs!

They also created a clever way to test the novelty of designs + compliance + cost → And then presented them in a clean fashion for designers to choose. The robustness of the model increases due to help from topological optimization!

Next steps is to advance to the realm of 3D (voxels), along with a recommendation system to predict the preference of designers and consumers.