Generating Physically Sound Designs from Text and a Set of Physical Constraints
Authors: Gregory Barber
Todd Henry
Mulugeta A Haile
Venue: NeurIPS 2025 poster ()
Keywords: generative design
structural optimization
text-image generation
differentiable physics simulation
Primary area: applications

Abstract: We present TIDES, a text informed design approach for generating physically sound designs based on a textual description and a set of physical constraints. TIDES jointly optimizes structural (topology) and visual properties. A pre-trained text-image model is used to measure the design's visual alignment with a text prompt and a differentiable physics simulator is used to measure its physical performance. We evaluate TIDES on a series of structural optimization problems operating under different load and support conditions, at different resolutions, and experimentally in the lab by performing the 3-point bending test on 2D beam designs that are extruded and 3D printed. We find that it can jointly optimize the two objectives and return designs that satisfy engineering design requirements (compliance and density) while utilizing features specified by text.

Generating Physically Sound Designs from Text and a Set of Physical Constraints
Gregory Barber, Todd C. Henry, Mulugeta A. Haile DEVCOM Army Research Laboratory Aberdeen Proving Ground, MD 21005
gregory.o j civearyy. miil
Abstract
We present TIDES, text informed design approach for generating physically sound designs based textual description and set of physical constraints. TIDES jointly optimizes structural (topology) and visual properties. A pre-trained text- image model is used measure the design's 's visual alignment with text prompt and a differentiable physics simulator is used to measure its physical performance. We evaluate TIDES on series of structural optimization problems operating under different load and support conditions, at different resolutions, and experimentally in the lab by performing the 3-point bending test on 2D beam designs that are extruded and 3D printed. We find that it can jointly optimize the two objectives and return designs that satisfy engineering design requirements (compliance and density) while utilizing features specified by text.
Introduction
Generative design has advanced rapidly and independently across the text-to-image (physics agnostic) and engineering (physics constrained) design domains. In this work, we bridge the two to address major challenge with each. First, text-image models [I+6] are powerful pattern generators and understand design concepts such as bilateral symmetry, asymmetry, and shapes such as hexagons, arches, and triangles. However, their application the domain of physical design is limited due to their inability to consider crucial physical design constraints, such as excessive structural deformation, material weight, and manufacturability. in the generated designs. That is to say, while they can generate artistic depictions of "a chair in the shape an avocado" they can not ensure the chair can support weight, is made of real materials, placing physical constraints on the output of a text-image generator ensuring the generated design is physically sound and validate by 3D printing and testing set of designs in the lab.
can be constructed. Here introduce method for
Second, structural optimization an approach for designing load-bearing structures such as support columns, beams, and bridges. Given a set of applied forces, materials, and supports, the goal is to generate the design that offers the greatest resistance to the applied force. However, it challenging to embed complex design goals into the design process, c.g. specifying the type of structural features present in the design such as arch, hexagonal, or triangular supports or an acsthetic goal. Here, we introduce method for using text-based guidance explore the design space and return diverse functional designs that utilize visual features specified by text. That is to say, we modify the structural optimization goal from generating the strongest possible design given set of physical constraints to generating the strongest possible design that meets an additional visual design criteria supplied by text, e.g. "Generate chair shaped like an avocado that supports the most weight". Additionally, unrelated text prompts have been found to increase originality in human generated designs (7]. We aim investigate if similar phenomenon be observed in algorithmic design approach.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

To address these challenges, we developed the Text Informed DESign (TIDES) approach. TIDES consists of three key components: design generator, a physics simulator, and a visual judge. The design generator parameterizes the design space and determines what material is placed at each point the design space. The physics simulator provides the means for evaluating the physical performance of the design, and its selection depends on the specific design task at hand. In the current work, we demonstrate on a fully differentiable structural design problem and use the differentiable finite element solver from [8) For non non-differentiable design tasks such as soft robotics [Р] or game level design [TO). TIDES can be adapted to use gradient free or genetic algorithm. The visual judge measures the perceived visual similarity of the design to a textual input. A pretrained CLIP model [11] is used as the visual judge. The goal of optimization in TIDES is to jointly maximize the perceived visual similarity of the design to textual imput and the physical performance of the design according to a selected performance metric such as structural stiffness and material cost.
"Leaning lower of Pisa"
"iffel
letters
TIDES designs
(our appreach)
(A)
Vision
designs
(H)
3631
Paysies alone
designs
(C)
V-29-25
v:25.1
11.06.56
v515.42
Figure 1: Tower design problem. The red arrow indicates the location a force is applied, and the blue line indicates the support the design rests on. The visual performance (v) of each design is given by the CLIP score. The physical performance is measured by the compliance (c), low compliance indicates greater resistance to the force. (A) Designs generated from our TIDES approach display features specified by text and resist the applied force. (B) Designs generated from a vision loss alone have understanding of physics indicated by floating material and orders of magnitude higher compliance. (C) Designs generated from a physics loss alone are physically sound and display simple solid features.
For example, consider the tower structural optimization problem given in Figure Here a load is applied from the top of the design space and the goal is to place the materials in the design space to carry the load with minimum deformation, i.e. maximize the stiffness of the design by minimizing compliance (c). The text prompts specified are: "Leaning tower of Pisa, dark black outline". "Eiffel tower, dark black outline", "Water tower, dark black outline" and "A stack of letters. dark black outline", and the goal is to embed features of these structures into the design. As a point of reference, we include the results for standard structural optimization, design optimized for physical performance alone in Figure TC and the results for maximizing vision only criteria (CLIP score) for a given text prompt in Figure IB. The physics alone designs utilize a simple solid support structure that does not resemble any of the text prompts, indicated by the low clip scores (v). The vision alone designs display features given by the text prompt, but have no understanding of physics indicated by the absurd compliance values and floating material. In Figure we plot the results for our TIDES approach for linking the visual and physical properties the design. Here, TIDES returns designs that utilize complex support features specified by text: the "Leaning tower of Pisa" design utilizes arch supports, the "Eiffel tower" utilizes a truss structure, the "Water tower" utilizes leg supports, and the "stack of letters" contains supports resembling letters, while resisting deformation from the applied force.
In summary our contributions are:

---

Physics constrained text-image generation: We develop a framework for placing physical constraints on a pretrained text-image generator to ensure the generated design physically sound and validate by 3D printing designs generated by our approach and testing them under load for a 3-point bending problem. This indicates it not necessary to train a new text to design model from scratch on physics data to generate sound designs.
Text informed co-design: We embed visual constraint into structural optimization, al- lowing complex design goals to be conveyed to the optimizer by text while respecting the underlying physical constraints. This is step towards co-design with text providing an intuitive means for shaping the structural support features.
Generating diverse physically sound designs: We find that varying the text prompt allows the generation of diverse designs that perform competitively in terms of structural compliance.
2 Background and Related Work
2.1 Structural optimization
Structural optimization is a specific field within topology optimization 12 concerned with the design of load-bearing structures such as support columns, beams, and bridges. One of the primary goals of structural optimization is to generate a design that can effectively withstand applied forees while minimizing the overall material cost. In this section, we provide a brief introduction to structural optimization. Introductions and tutorials in structural optimization [[3] exist for readers unfamiliar with the domain who are looking for a more thorough introduction.
structural optimization problem is defined by four factors. The first is the design space. The design space bounds the design and is specified by the dimensions of the design problem. For computational purposes this space divided into a grid of cells called finite elements. At every cell in the design space a material can be present or not. The second factor is the location of the fixed support points or normals the design will reset on. The third factor is the location the load or force is applied at. The fourth factor is the material cost. This penalizes the amount of material used in the design process and serves two purposes. The first to generate non-trivial solutions. As we are maximizing the structure's ' stiffness the optimal solution is often to simply fill the entire design space with material. To generate more interesting designs, we must then limit the material through a cost penalty. The second is to reduce construction cost, A cost-effective structure will use the minimum amount of material to achieve a desired stiffness. Together these factors along with the choice in optimizer and random seed shape the design.
In practice it is common to work with continuous values for material density, d [0. 1]. rather than binary variables for material presence or absence at each element. This has the advantage of allowing for gradient based optimization methods. Denser elements can be viewed as "more" present and offer greater contribution to the structures stiffness. After optimization, when looking to assemble the design elements with densities close to zero will be treated as voids (no material present) and values close to one will be filled with material. Thus during optimization it is key to push density values towards or 1. The Solid Isotropic Material with Penalization method (SIMP) 15 16] a widely used approach to solve structural optimization problems. This approach operates on continuous
densities E [o. 1] and contains a penalty term p = to steer elements towards or 1. In this work we use the modified SIMP equation from [17].
(1) Where E. is the stiffness coefficient or Young's modulus of the material at each element. Emin is the minimum allowed stiffness of any element and will indicate voids in the final design. Eo is the Young's 's modulus of solid material. The goal in optimization is to maximize the stiffness of design subject to a material cost and the application of a force. The parameters are the densities (presence of material) at each element. To evaluate the performance of the design we compute the design's 's elastic potential energy or compliance c, given in equation?? 2
(d.) = Emin +
Emina)
c(d) = U" = Σ (de) kouu
Where U is the displacement vector, K is the global stiffness matrix and KU = F where the vector of applied forces. Compliance measures the structures displacement or deformation from the
3

---

force. A stiffer design will have a smaller compliance. In optimization we minimize the compliance to generate stiffer structure. To impose material cost m we take a simple approach and add an additional term to the loss given by the MAE between the design's density d and some specified target density d², This approach was chosen to allow independent weighting of the visual, physical, and material cost components of the total loss equation: 5
m
ld" d|
The optimality and functionality of the generated design are important considerations. "Topology optimization problems have extremely many local minima" ([88.. This make it challenging to arrive at the globally optimal material layout for a given design problem. In the current work our focus is on generating functional designs rather than optimal designs. A functional design is design that resists deformation and can be specified by variable performance target for example a compliance threshold. For given threshold there can be multiple possible functional designs. Generating and exploring the space of functional designs has previously been explored in the context of generating diverse competitive designs 19 and architectural design (20].
2.2 Generating art with CLIP
In this section, we provide a brief introduction into generating art with CLIP. CLIP I1] is a model for connecting images and text. Its architecture consists of an image encoder and text encoder. During training the encoders are respectively fed images and their corresponding textual description, and the weights are optimized maximize the cosine similarity between the text and image embeddings. After training the weights can be frozen and CLIP can be used to measure text image alignment. For example, given an image of a dog and two textual descriptions ["a dog" and "a cat"] we would expect the similarity between the image embedding and text embedding for "a dog" to be higher than the similarity of the image embedding and text embedding for "a cat".
We can then think of CLIP as a visual judge. Given some image and text CLIP can be used to score the perceived visual similarity of the image to the provided text. If we now hold the text description constant and replace the image with a blank canvas parameterized by learnable weights, we have the setup an image generation problem. By maximizing the cosine similarity and backpropagating through the frozen CL.PP image encoder we can update the canvas to generate images with features specified in the textual description. This approach has been widely used to generate artistic images and designs. Deep Daze [21) generates images by updating the parameters of SIREN an implicit neural representation network (22]. CLIPDraw 23 generates images with a differentiable render and learning vector stroke arrangements. [24] generates images using an evolutionary search algorithm and CLIP loss to determine the placements of shapes. CLIP has also been used to generate diverse
3D objects 27
2.3 Physics-based constraints for visual/physical design
Recent efforts have explored physics-based constraints for visual/physics design problems. Several approaches have focused on generating designs that maintain stability under gravity. Atlas3D [28) refines designs generated by a diffusion model using physics-based loss formulated for standability. DSO (29) assembles a dataset of 3D objects and their stability scores and fine-tunes a diffusion model on this dataset. 30 uses an input image guide the visual geometry of 3D objects and physics constrained optimization for stability and soft object deformation. Our approach differs in that our focus is on a different design problem, structural optimization, where the forces applied can vary in intensity and placement throughout the design space. Additionally, our approach does not require starting from an existing image or design generated by a diffusion model. Other efforts have looked to data-driven methods to incorporate differentiable physics constraints. Phy3DGen [31] trains a neural network to approximate a solid mechanics finite element solver. DrivAerNet++ (2) introduces a large multimodal dataset for acrodynamic car design for surrogate modeling and data-driven design. Our approach does not require data as the physics is differentiable.
In parallel to this work (33 examined CLIP stylization for topology optimization. However, the approach taken in (33 utilizes RGB channels and connected component labeling, which can cause overfitting to the visual domain. Our approach differs in that we do not use RGB values and the focus is on generating a shared binary (material presence/absence, black or white pixel) distribution. Additionally, introduce an efficient physics based approach for removing unattached material

---

rather than connected component labeling. In Appendix Figure 9 we provided a comparison with our approach for an deco building. In the design generated by 331 the windows exist only in RGB space and are painted over solid material, whereas in the design generated by TIDES the windows are structural features where there is no material present.
3 Text informed design
In Section 2, we provided background into two design tasks: Structural optimization a physics grounded approach for generating designs that resist deformation from an applied force and Gen- erating art with CLIP guidance an artistic approach for generating visually interesting output from text. We argue that there is natural overlap between these two design tasks. Both operate on grid of pixels/elements and can be optimized with a differentiable performance metric. It then may be possible share a grid of pixels/clements across both tasks and jointly optimize the design to maximize both its visual and physical performance.
We pose that linking design's 's physical performance and visual appearance may offer number of benefits:
Text to drive design diversity and navigate the space of possible designs. In standard topology optimization diverse designs are generated through randomizing diversifying initial conditions. The suitability of the initial conditions based on the shape of the design space which may or may not be known apriori. Text guidance may allow for the use of prior knowledge of structurally strong shapes given as text e.g. an "arch" or "hexagon" to steer the design process and reduce the number of runs optimizer needs to reach fit solution. Further, text guidance may provide a means of intelligently exploring a reduced set of initial conditions in design space instead of relying randomness drive design diversity, e-g- in Figure 4C and D, we plot competitively performing designs that use different support strategies specified by text.
2. Complex feature specification. In standard topology optimization approach, is not clear how one would include complex visual design criteria in the optimization e.g. generating tower that both structurally sound and "resembles a robot", A visual loss specified by text may provide mechanism for complex feature specifications.
3. Mechanism for applying location based attention. The placement of material in the image/design space depends on the locations a user has specified for the support and the force application. From an artistic perspective this could be used to ground the placement of aesthetic features and create focal points in the image where material must be placed to resist physical deformation. E.g. given an RGBA image, the alpha channel could define a structural optimization problem and be jointly optimized with visual loss across the RGB channels.
In the current work, our focus is on the first two. However, to bridge design's physical and visual performance there are two challenges we need to address. The first is mapping between the visual and material domain. CLIP was trained on 3 channel 224 by 224 RGB images while structural optimization operates on 1 channel densities bound between 0 and I where the design space may take various shapes and sizes. To resolve the size and shape differences we resize the design with bilinear interpolation. To resolve the channel difference, we repeat density channel 3-fold to generate a grey scale image. The inclusion of the visual loss term introduces an additional issue in that visually appealing images often make use of artistic techniques, such as shading, where pixel values spread between 0 and 1. This is not ideal for structural optimization where it is desired for the pixels to occupy values close to 0 or 1, corresponding to material presence or absence.
To address this challenge, we introduce a Hill function-style sigmoid equation and apply it to the output of the design encoding to push the densities close to 0 or 1. In Appendix C.2 Figure 10, we include examples of designs generated using our Hill function and with the Hill function removed. The Hill function removed designs contain intermediate grey scale values while the Hill function designs utilize values close to 0 and indicating the Hill function is successful in pushing the densities to 0 or 1. The Hill function style sigmoid employed is given in equation 4 where the
5

---

values a 0.8 and n 20 were determined heuristically.
+0.1
Further while CLIP was primarily trained on RGB images the training data includes images that are binary (black and white) in nature e.g. (silhouettes, outlines, shadow art, line art, and stencils). We append such terms to the end of the text prompts.
The second challenge is to ensure that all pixels contribute structurally to the design. The introduction of a visual loss term can lead to the creation of aesthetic features in the design that do not offer structural support. These features are often unattached to the main structure and float in space e.g. the "clouds" in Figure ITC. To solve this issue, we developed a compliance based masking approach to trim non-structural features from the design prior to showing it to CLIP. The compliance mask is generated by thresholding the element wise compliance returned from the physics simulator i.e. mask = log( (og(complia 2 threshold; threshold = -20. This returns a mask of zeros and ones where zero indicates given pixel does not contribute structurally and one indicates it does. The mask is applied by multiplying by the design encoding. This ensures the CLIP loss only reflects pixels directly contributing to support and prevents over fitting to the visual loss through the creation of solely aesthetic features. In Appendix C.2 we generate designs with and without the compliance mask. The designs generated with the compliance mask contain no floating material while the designs generated with mask removed contain floating material indicating the masking approach ensures all pixels are connected to the design. In Figure 2 we sketch TIDES our text informed design framework. TIDES consists of three key components: design encoding, physics simulator, and visual judge.
Visual
CL
Augerest
enoode
Corpl
Las
Maysics
Figure 2: The proposed text informed design framework. The framework jointly optimizes a text to image visual loss, a physic- based loss, and a material cost.
The design encoding parameterizes the design space. Here, we modify a direct encoding approach commonly used in structural optimization. In this approach, there is one parameter for every element in the design space. At initialization, every parameter is assigned a value. The starting parameters can be a random canvas, zeros for an empty canvas, or ones for a solid canvas. We opt for a starting value of as we can think of this as starting from solid block of material. A Gaussian blur filter
then applied across the parameter grid. The filter is used to avoid checkerboard artifacts 34 In
standard structural approach these densities would then be passed into the SIMP equation I Here we add the additional step of applying the Hill function-style sigmoid equation to return the densities and push the densities towards or
The visual judge receives the output of the design encoding as input. A compliance mask is applied zero out sections of the design that do not contribute structurally. The design is then repeated 3-fold channel wise to return a grey scale image. The image is then resized to 224 by 224, the dimension expected by the CLIP image encoder. The image augmentation scheme from 23 is then employed to generate a batch of images. The image is randomly cropped, random perspectives are taken, and the images are all resized. These operations are performed using the Viscon library (35). The batch of images is then passed through the CLIP Image Encoder, and the cosine similarity is computed between the image embedding and the text embedding. Randomly sampling sections of the images has the benefit of allowing TIDES to focus on the visual aspect of different sections of the design between trials. This enables TIDES to generate different designs when given the same prompt and problem setup in repeated trials.

---

The physics simulator receives the output of the design encoding as input and executes the modified SIMP approach, equations I and 2 to compute the compliance. We use the AutoGrad (36 SIMP
implementation from g with an added wrapper to pass the gradients from autograd to pytorch.
The material cost is computed as the mean absolute error between the target density and the current density. Intuitively, we can think of the target density as the percentage of the design space the design should occupy if all pixels are either 0 or 1.
The loss function for TIDES is given in equation 5 Where m is the material cost, is the compliance. is the vision loss given by clip score and 31 and Bg are heuristically determined values for weighting the material cost and visual loss.
L = c Bim- - B2
(5)
4 Experiments
In this section, we present experimental results for designs generated with TIDES. We include high-resolution plots of all designs in Appendix([along with detailed descriptions of the methods and experimental setup in Appendix The results cover structural design problem classes including single point force application, multiple point force application, multiple points and varied force application, and for 3D printing the 3-point bending problem used to test for beam strength. Additionally, in appendix [..6] we include results for starting from and tuning an existing design to improve physical performance.
4.1 Designs generated at different resolutions
Design resolution: x
128 128
256 x 256
512 x 512
aarm chair in the shape of an avocado, dark black solid outline" (A)
1:.55
29.39
33.994
"art deco hotel, dark black solid outline"
(B)
15.16
Figure 3: Generating designs at increasing resolutions for a tower structural optimization problem. A force is applied at the center the design space indicated by the red arrow and the design rests on support given by the blue line. As the resolution increases feature complexity increases.
The pretrained visual judge (CLIP) was trained on 3 channel fixed resolution 224 x 224 images, while the design space in structural optimization operates on channel and may take on various shapes and sizes. To evaluate TIDES ability to map between the visual and material domains and scale to structural design problems below or above the fixed resolution input of CLIP, designs were generated for tower structural optimization problem at increasing resolutions from 32 x 32 to 512 x 512.
In Figure AA we plot designs generated from the text prompt: "arm chair in the shape of an avocado, dark black solid outline".. Here, TIDES returns designs that increase in feature complexity with resolution. At 64 x 64 the returned design is a simple silhouette outline of a chair with bulbous shape, at 128: x 128 texture emerges in the form of small cut outs and by 512 x 512 the design captures the pitted skin surface of an avocado. The designs are composed of two regions; the round bulbous shape occupies the top half of the design and the stand-like structure on which it rests at the bottom. The compliance of the TIDES designs are orders of magnitudes lower than the vision alone results Appendix C.3 Figure ITA and are within the same magnitude of the physics alone results Appendix C.3 Figure ITC. indicating the designs are physically sound while utilizing complex structural support features supplied by text.
C.3
7

---

In Figure 3B, we plot designs from the text prompt "art deco hotel, dark black solid outline". Here, feature complexity again increases with resolution. At 64 64 the design returned is simple rectangular shape with square window-like cutouts, at 128 x 128 the detail increases and the design resembles the triangular shape of the famous Flatiron building in New York and by 512 x 512 the tower rises above detailed marquee. The designs share the perspective of an individual looking up at the tower. The compliance the TIDES designs are orders of magnitudes lower than a vision alone result Appendix C.3 Figure II and are within the same magnitude of the physics alone results Appendix C,3 Figure ITC while utilize complex support features supplied by text. This suggests that the mapping approach and Hill function design encoding employed in TIDES allow for joint optimization of a design's 's visual and physical properties across different scales.
4.2
Navigating the design space with text
text_prompt
large hexzgon, dark slack stldd outtine
aarge triangle. dart stack sold outline
large arch, dar blnck solid outfin
CUP score
Score M
(A) Suspended bridge: #30 trials
(8) Hoop: #30 cnals
"Hreagor
"MWongge"
"Tranyle
Tot
"Avrin
Plycis alone
(C) Sucpended bridge designs
Hoop designs
Figure 4:
Distribution of design performance across 30 trials. (A) and (B) results for a suspended
bridge and hoop design problem, the 90% confidence ellipse is plotted for each trial. (C) and (D)
examples designs returned. All designs for each trial can found in the Appendix C.5
and C.6
For given design problem, there are multiple possible paths an optimizer can take to achieve functional design. To assess text as a means of navigating the design space and generating diverse designs, we generated designs for suspended bridge and hoop design problem, Appendix B..
B.3 30 trials of the TIDES approach were conducted for three different text prompts selected to elicit structural support features: large triangle, dark black outline". "a large hexagon, dark black outline",, and "a large arch, dark black outline".. Additionally, 30 trials were conducted for physics loss alone, where the vision component was removed and the starting canvas was randomized to generate different designs between trials.
In Figure 4A & B, we plot the resulting design distributions. We plot design for each text prompt trial in 4 & D, a plot of all designs can be found in Appendix C3.5 & C.66 The designs returned for the suspended bridge and hoop design problems visually display and structurally utilize the support strategies specified by text, c.g. in Figure 4C & D: the designs generated from the "Hexagon" prompt utilize hexagonal mesh, the designs generated from the "Triangle" prompt utilize large central triangle and surrounding smaller triangles, and the designs for the "Arch" prompt utilize large central arch and surrounding smaller arches. For the suspended bridge design problem we observe similar physical performance across designs utilizing different support strategies and features specified by text, with designs generated by TIDES approaching the upper bound of the physics alone results in terms of compliance suggesting that designs resist deformation. For the hoop design problem, the designs returned for the arch and triangle prompt overlapped performance with the physics alone results. This is point of interest, it is expected that inclusion of visual constraint

---

into the structural design problem will lead to some trade off in physical performance and suggests in some cases the addition of visual text based loss may act as a mechanism for dislodging designs stuck at a local minimum.
4.3 Diverse design problems and text prompts
Text
Design problem
Vision alone
Plysics alone
TIDES
nobsr, dark baak outline
burhee, dark
"s skrimon doakesl Mookae dark Mack outlie"
la gieat dispks prohing
bealdes dan black octline
"aquare bencs, dark blak curline"
"a casi reef, dark block outine"
Figure 5: Generating designs for diverse initial conditions (force and support positions and text prompts). The designs resist deformation from the applied force while displaying features specified by text suggesting TIDES is robust to changes in the initial conditions. In Appendix: B.5 we provide the configurations for each design problem.
As input, TIDES receives text prompt, target density, and the locations of the supports and applied forces. Figure 5 we plot designs generated for a series of different text prompts and support and force positions. Across all structural design problems and text prompts: the vision alone generated designs display the requested visual feature but have no understanding of physics, the physics alone generated designs resist deformation and utilize simple support strategies that have no understanding of the visual design goal, and the TIDES generated designs are both physically sound, compliance values within the same order of magnitude of the physics alone results, and utilize complex support features specified by text. This indicates that TIDES is robust to changes in the initial conditions.
4.4 3D printed beams for a 3-point bending problem
The experimental performance of the TIDES approach was assessed by 3D printed beams subjected to three point loading. The 3D printing and testing setup is described in Appendix B.4 In FigureloA, we plot the final designs generated by binarizing the TIDES and the physics alone output densities to 0 or 1. 6 we plot an example of the beams returned from the 3D printing process, 3 replicate beams were produced and tested for each design to account for variations in the printing process. In general, the rather fine features presented in this manuscript lead to some flaw development associated with terminations in paths of material when any dimension is not discrete numeral of the path width (200 microns). At locations where this occurs void content will create stress concentrations and reduce part stiffness and strength. The x-direction strain field (left-right in the image) is presented in Figure 6D at a displacement of around 2.66 mm which about the onset of non- non- lineariyy which can be seen in Figure 6C. The magnitude of the maximum measured strains is of the same order with the distribution and location of maximal values dependent on the morphology.
From the force-displacement graph, Figure the physical performance of the TIDES and Physics alone are the most similar in the 1.5 mm region of loading with some divergence from 1.5 to 2.66 mm. The physics alone design had the lowest compliance value of 1.0e-3 N/mm followed by

---

Physics alone
large arch
honeycomiss
many bexagons"
(A)
Generated design
(B)
3D peinted beams
Figure 6: 3D printed beams tested on a 3 point bending problem. (A) Generated designs from TIDES and physics alone. (B) 3D printed beams returned from extruding the design along the y-axis. (C) Force- dirreeeippaaceent behavior from the point bend testing. The points correspond the mean force and the bars indicate the minimum and maximum force value observed across the three replicates per trial. (D) Snapshot of beams under load. The beams rests on two separated points and force is applied at the center.
the "honeycomb, many hexagons" with 1.13e-3 N/mm and the "large arch" 1.19e-3 N/mm. This aligns with the simulation results where the physics alone result returned the lowest compliance value of 280.0 c, note the simulation compliance values have different unit than the experimental, in simulation the compliance is computed for every element and the summed value is reported and experimentally displacement is measured for the marked points in Figure 6D (L1-L2) and used to compute compliance. In simulation the "large arch" design returns slightly lower compliance value of 285.71 to the "honeycomb, many hexagons" design's 's 299.55 this differs from the experimental results where the "honeycomb, many hexagons" design returns a slightly slower compliance than the "large arch" design. This could be due manufacturing previously mentioned. The performance across both the experimental and simulation for all trials are within the same order magnitude and indicate that all designs successfully resist deformation from the applied force.
5 Discussion and Conclusion
In this work, presented TIDES, approach for bridging the design domains of physic agnostic text-to-image generation and physics constrained structural optimization. To the best our knowl- edge, this is the first approach to jointly optimize visual and physics- loss for load bearing design problem. Experimental results across a series of text prompts and structural optimization problems operating under different load and support conditions, different resolutions, over repeated trials, and experimentally in the lab through 3D printing, show TIDES generates designs that resist physical deformation from an applied force while utilizing support strategies and features conveyed by text and robust to changes in the initial conditions. This suggests that placing physical constraints on pretrained text-image model is valid new approach for ensuring that the generated design is physically sound and indicates that it is not necessary to train a new text to design model from scratch on physics data generate sound designs. Further, from a structural design perspective, it suggests that embedding visual constraint into structural optimization allows for text informed co-design and diverse design generation and a valid mechanism for conveying complex design goals, such as using a specified structural shape, e.g. "Hexagon", "Arch", and "Triangle" in the support strategy or more aesthetic driven goal such as "in the shape of an avocado" or "art deco hotel", while respecting the underlying physical constraints. We believe that linking text-image models and differentiable physics simulators an exciting new area of research towards generating physically sound designs that can be realized in life.
10

---

References
[1] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International Conference on Machine Learning, pages 8821-8831. PMLR, 2021.
[2] Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Πya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112. 10741, 2021.
[3] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemmpour, Bureu Karagol Ayan, Sara Mahdavi, Rapha Gontijo Lopes, et al. Photorealistic text-to-imate image diffusion models with deep language understanding, arXiv preprint arXiv:2205.11487, 2022.
[4] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 10684-10695, 2022.
[5] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 2022.
[6] James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dallle-3 pdf. 2023.
[7] Gabriela Goldschmidt and Anat Litan Sever. Inspiring design ideas with texts. Design studies, 32(2):139-155, 2011.
[8] Stephan Hoyer, Jascha Sohl-Dickstein, and Sam Greydanus. Neural reparameterizatioo improves structural optimization. arXiv preprint (1XXvx:909.04244: 2019.
[9] Nick Cheney, Robert MacCurdy, Jeff Clune, and Hod Lipson. Unshackling evolution: evolving soft robots with multiple materials and a powerful generative encoding. ACM SIGEVOlution, (1):11-23, 2014.
[10] Sam Earle, Justin Snider, Matthew C Fontaine, Stefanos Nikolaidis, and Julian Togelius. Illuminating diverse neural cellular automata for level generation. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 68-76, 2022.
[11] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, pages 8748 8763. PMLR, 2021.
[12] Martin Philip Bendsoe and Ole Sigmund. Topology optimization: theory, methods, and applica- tions. Springer Science & Business Media, 2003.
[13] Sam Greydanus. A tutorial on structural optimization. arXiv preprimt (.XXi:22005.0966 2022.
[14] Erik Andreassen, Anders Clausen, Mattias Schevenels, Boyan S Lazarov, and Ole Sigmund. Ef- ficient topology optimization in matlab using 88 lines of code. Structural and Multidisciplinary Optimization, 43:1-16, 2011.
[15] Martin Philip Bendsøe and Noboru Kikuchi. Generating optimal topologies in structural design using a homogenization method. Computer methods in applied mechanics and engineering, 71 (2):197-222) 224. 1988.
[16] M Zhou and GIN Rozvany. The coc algorithm, part ii: Topological, geometrical and generalized shape optimization. Computer methods in applied mechanics and engineering, 89(1-3):309-33 1991.
[17] Ole Sigmund. Morphology-based black and white filters for topology optimization. Structural and Multidisciplinary Optimization, 33:401-424, 2007.

---

[18] Ole Sigmund and Joakim Petersson. Numerical instabilities in topology optimization: survey on procedures dealing with checkerboards, mesh-dependencies and local minima. Structural optimization, 16:68-75, 1998.
[19] Bo Wang, Yan Zhou, Yiming Zhou, Shengli Xu, and Bin Niu. Diverse competitive design for topology optimization. Structural and Mulridisciplimary Optimization, 57:891-900 1-92,, 2018.
[20] Yi Min Xie. Generalized topology optimization for architectural design. Architectural Intelli- gence, 1(1):2, 2022.
[21] Ryan Murdock. Deep daze. http://ggrraaa..com/uuccdaadddee
2021.
[22] Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein, Im- plicit neural representations with periodic activation functions. Advances in Neural Information Processing Systems, 33:7462-7473, 2020.
[23] Kevin Frans, Lisa B Soros, and Olaf Witkowski. Clipdraw: Exploring text-to -atttonn synthesis through language-iman image encoders. arXiv preprint 1rXxi:21006118433 2021.
[24] Yingtao Tian and David Ha. Modern evolution strategies for creativity: Fitting concrete images and abstract concepts. In International Conference on Computational Intelligence in Music, Sound, Art and Design (Part of EvoStar), pages 275-29) Springer, 2022.
[25] Ajay Jain, Ben Mildenhall, Jonathan T Barron, Pieter Abbeel, and Ben Poole. Zero-shot text-guided object generation with dream fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 867-876, 2022.
[26] Jiale Xu, Xintao Wang, Weihao Cheng, Yan-Pei Cao, Ying Shan, Xiaohu Qie, and Shenghua Gao. Dream3d: Zero-shot text-to-3d synthesis using 3d shape prior and text-to-image diffu- sion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20908 20918, 2023.
(27] Pengkun Liu, Yikai Wang, Fuchun Sun, Jiafang Li, Hang Xiao, Hongxiang Xue, and Xinzhou Wang. Isotropic3d: Image-to-3d generation based on a single clip embedding, arXiv preprint arXiv:2403.1032 10395, 2024.
[28] Yunuo Chen, Tianyi Xie, Zeshun Zong, Xuan Li, Feng Gao, Yin Yang. Ying Nian Wu, and Chenfanfu Jiang. Atlas3d: Physically constrained self-supporting text-to-3d for simulation and fabrication. Advances Neural Information Processing Systems, 37:102501-102530 2024.
[29] Ruining Li, Chuanxia Zheng, Christian Rupprecht, and Andrea Vedaldi. Dso: Aligning 3d generators with simulation feedback for physical soundness. arXiv preprint 2:X2::2503.22677, 2025.
[30] Minghao Guo, Bohan Wang, Pingchuan Ma, Tianyuan Zhang, Crystal Owens, Chuang Gan, Josh Tenenbaum, Kaiming He, and Wojeiech Matusik. Physically compatible 3d object modeling from a single image. Advances in Neural Information Processing Systems, 37: 37:1 11260- 2024.
[31] Qingshan Xu, Jiao Liu, Melvin Wong, Caishun Chen, and Yew-Soon Ong. Precise-physics driven text-to-3d generation. arXiv preprint arXiv:2403. / /2438, 2024.
[32] Mohamed Elrefaie, Florin Morar, Angela Dai, and Faez Ahmed. Drivaernet++: A large- scale multimodal car dataset with computational fluid dynamics simulations and deep learning benchmarks. Advances in Neural Information Processing Systems, 37:499-5366. 2024.
[33] Shengze Zhong, Parinya Punpongsanon, Daisuke Iwai, and Kosuke Sato. Topology optimization with text-guided stylization. Structural and Multidisciplinary Optimization, 66(12):256, 2023.
[34] Blaise Bourdin. Filters in topology optimization. International journal for numerical methods engineering, 501 5009):2143-21555 2001.
[35] Torch maintainers and contributors. Torchvision: Pytorch's computer vision library.
http://gg.ttabb.cm///ooorrrr//aa 2016.
12

---

[36] Dougal Maclaurin, David Duvenaud. and Ryan P Adams. Autograd: Effortless gradients in numpy. In ICML 2015 AutoML workshop, volume 238, 2015.
[37] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017.
[38] Brad D Lawrence, Michael D Coatney, Francis Phillips, Todd C Henry, Yuri Nikishkov, and Andrew Makeev. Evaluation of the mechanical properties and performance cost additively manufactured continuous glass and carbon fiber composites. International Journal of Advanced Manufacturing Technology, 120: 1 135-1147, 2022.
[39] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björ Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10684 1004-1009,,,,00 June 2022.
13

---

Appendix
A Limitations and Impact Statement
This paper presents work whose goal to advance the field of generative design by bridging the domains of physics constrained structural design and unconstrained text-image generation. As TIDES generative framework that uses CLIP, a pretrained text-image model to guide the designs produces, there the inherent risks of misuse and the amplification of biases present in the training data. Our approach has the added risk of allowing the generation of load bearing designs that could be constructed through means such 3D printing. Our approach is currently limited by the simulation environment restricting 3D designs to layering of 2D force applications. Future work could extend the approach simulators that support 3D meshes and higher resolution solvers and could have impacts on 3D design domains such as engineering, sculpting, and architecture.
B Design problem details and methods
B.1 Tower design problem
In the tower design problem, Figure: D the design rests on the ground and resists deformation from a downward force applied the top. target density 0.3 is selected and the x and y dimensions of the design space are set at 128. As the force application is symmetric, we compute the compliance for half of the design and mirror the results in the image shown CLIP. We used the text prompts: "I "Leaning tower of Pisa, dark black outline", "Eiffel tower, dark black outline", and "a stack of letters, dark black outline".. In the loss function 31 50 and 83 100. The Adam'W (37 optimizer was used with a learning rate of 0.25 and train for 100 epochs. Lastly we perform a quick manual pass to remove any disjoint artifacts created by the compliance masking, these disjoints are connected to the main structure, do not contribute structurally and occur where the mask occludes pixel with compliance over specified threshold. In the varied resolution tower design problem, Figure: 3 the same setup is used and the design space is set to (64 x 64), (128 128), (256 256) and (512 x 512). The text prompts given tides are as follows: "arm chair the shape of an avocado, dark black solid outline" and "art deco hotel, dark black solid outline"..
B.2 Hoop design problem
Figure 7: Hoop design problem setup
For the hoop design problem, Figure: 43 ADd the force and support layout given in Figure: the design rests on the ground and a variable a downward force is applied around the hoop. A target density 0.3 is selected and the the a and dimensions of the design space are set 128. As the force application is symmetric, we compute the compliance for half of the design and mirror the results in the image shown to CLIP. We use the text prompts: "a large triangle, dark black outline",, large hexagon, dark black outline",, and "a large triangle, dark black outline". In the loss function 31 = 50
14

---

and B2 = 100. The Adam W [7 epochs.
optimizer was used with learning rate of 0.25 and train for 100
B.3
Suspended bridge design problem
Figure 8: Suspended bridge design problem setup
For the suspended bridge design problem, Figure: 4 and C, the force and support layout is given in Figure: 8 the design rests on separated points and downward force is applied along the span of the bridge. target density 0.3 is selected and the dimensions of the design space are given as 256 and y 128. As the force application is symmetric, we compute the compliance for half of the design and mirror the results in the image shown to CLIP. We use the text prompts: large triangle, dark black outline", "a large hexagon, dark black outline", and "a large triangle, dark black outline".. In the loss function 31 = 50 and B3 100. The Adam [37] optimizer was used with learning rate of 0.25 and train for 100 epochs.
B.4 3D printing and testing
For the beam design problem. Figure: GA. the force and support layout is given in Figure: 6C the design rests on two separated points and force is applied at the center. A target density 0.5 is selected and the dimensions of the design space are given x 672 and y 96. This gives (7 1) inch ratio in the 3D printed design. As the force application is symmetric. we compute the compliance for half of the design and mirror the results in the image shown to CLIP. We use the text prompts: "a large arch, dark black outline" and "a honeycomb, many hexagons, dark black outline".. In the loss function 81 50 and 83 100. The Adam [37 optimizer was used with a learning rate of 0.25 and train for 100 epochs.
Beams were printed using a Markforged X7 3D printer which extrudes nylon highly filled with short carbon fibers trademarked Onyx. Parts are build layer by layer starting with the paths of outer shell material infilling the center volume at alternating +45 and -45 deg. path of material has a finite minimum width of 200 microns and a height of 100 microns limiting the minimum feature size. The beam rests on two lower rollers which move upwards in a displacement control test mm/min while the upper center roller is fixed. The center fixture has 20 kN load cell that is used to measure the applied load. A more detailed description of the setup and methods are available in [8
B.5 Diverse design problems
For the roof design problem, the target density was 0.3 and the dimensions of the design space were 128 and y 64. In the loss function 31 = 10 and 32 250. For the staircase design problem, the target density was 0.4 and the dimensions of the design space were x = 64 and y 64. In the loss function B1 10 and 3g 250. For the cantilever beam two design problem, the target density was 0.5 and the dimensions of the design space were = 80 and y = 64. In the loss function 31 100 and B₂ 250. For the dam design problem, the target density was 0.5 and the dimensions of the design space were = 64 and multistory building design problem, the target density was 0.5 and the dimensions of the design space were x = 70 and problem, the target density was 0.5 and the dimensions of the design space were x 80 and y = 80. In the loss function 3 = 50 and 83 = 50. The Adam W optimizer was used with a learning rate of 0.25 and train for 100 epochs for additional design problems. The adjustments here to the weights on the visual loss and material cost for each trial were made to balance out an increase in the initial
80. In the loss function 83 10 and B3 = 250. For the
64. In the loss function 31 50 and 83 250. For the staggered point design
15

---

physics loss in some of the more complex design problems. In B.1-B.4 the same weights (81 = 50 and B2 100) are used for all trials.
B.6 Starting from existing design
Stable Diffusion 2.1 139 was used to generate a 768 x 768 image for the text prompts: "the eiffel tower, dark black outline", "seattle space needle, dark black outline", "a robot, dark black outline", and "st louis arch, dark black outline". The default settings for Stable Diffusion 2.1 were used. The returned images were processed for compatibility with TIDES design encoding. The images were converted from three channel RGB to one channel grey scale and resized to 256 x 256 to match the design space of the structural optimization problem. This was then passed as the starting parameters for TIDES design encoding. A tower design problem B.I was used. The target density and learning rate were adjusted account for starting from an existing design. The learning rate was reduced and the target density was selected as a value less than the mean density of the starting image to account for the amount of unconnected material in the starting design that will be removed during optimization. For each text prompt the target density and learning raters given as "the eiffel tower, dark black outline". learning rate 0.02 and target density = 0.4, "seattle space needle, dark black outline": learning rate = 0.05 and target density 0414, "a robot, dark black outline": learning rate = 0.1 and target density 0.4, and "st louis arch, dark black outline": learning rate 0.02 and target density 0.2. All other parameters used the same settings as B.1
C Additional Results
C.1 Comparison with 33
(Zhong, 2023)
TIDES "Art Deco"
"Att Deco"
Windows
Windows
RGB
Material layout
layout
A) Zhong et al., 2023
B) TIDES
Figure 9: Comparison with 333 for an "Art Deco" structure. (A) 33 utilizes RGB channels and can overfit to the visual domain e.g. the generated Art Deco building has off-white colored windows. These windows are not structural features and are painted over solid material. (B) TIDES does not use RGB values, the focus is on generating a shared binary (material presence/absence, black or white pixel) distribution. The window features present in TIDES all correspond structural features (no material is present).
In 33) RGB channels are utilized during the design generation process. This can cause overfitting to the visual domain by allowing the model to improve the loss by painting over sections of the underlying structure rather than by shaping the physical structure itself. E.g. Figure DA, the generated art deco building has off-white colored windows. These windows are not structural Teatures and are painted over solid material. Our work does not use RGB values, the focus is on generating shared binary (material presence/absence, black or white pixel) distribution. The visual loss tends to
16

---

favor generating artistically pleasant images that utilize shading rather than binary values. To solve this, we introduced Hill-function into the design encoding control material distribution and push pixel/density values towards 0 or 1. E.g. In
B, we present the design generated by our approach for an art deeo building. In contrast to [3,, the windows our design all correspond to structural features (no material is present). 33 does not scale well to large design spaces and can result in designs with features that do not offer structural support. (33] utilizes the Connected Component Labeling (CCL) algorithm to remove unconnected features, this is computed at every optimization step. CCL scales poorly in computational time as the size the design increases, In our work, we introduce a compliance based masking approach to remove unconnected features. As compliance is already computed for structural optimization there no additional computational overhead required. Further compliance is physies based and allows the removal of features that are not offering direct structural support.
C.2 Hill function and compliance masking ablation
TIDES
Compliance mask removed
Hill function removed
Compliance musk and Hill finetion removed
dark black cotline'
33.42
31.35.3
34.99
v:
Hill function and compliance masking ablation. The compliance masking approach developed for TIDES ensures all features are connected. Removing this mask leads to designs with floating material. The Hill function developed for TIDES pushes density values to 0 or 1 (material presence or absence). Removing the Hill function leads to designs that utilize intermediate grey scale values between 0 and 1. See Figure 1 and Figure I1 for examples of designs generated from ablating the vision and physics loss components from TIDES.
Figure 10:
C.3 Designs generated multiple resolutions
64 64
128 128
256 256
512 x $12
"arm chair the shape of an avocado, dark black solid outline" (A)
et 6-4.32+009 v:. $1.28
e: 4.0220009
e: 1.722-00
6.
vi 34
en $3.19
Vision loss alone
bilte offlic
"art deco hotel, dark black solid outline"
(B)
vn 32.59
34.77
78
Physics loss alone
Physics alonc
(C)
Figure 11: Loss ablation. Optimizing the tower structural optimization problem in Figure 3 for only physic vision loss. (A) and (B) are designs generated from maximizing the CLIP score. These designs have concept of physics and return designs that have a compliance orders of magnitude higher then a sound design, and contain floating material. (C) The designs returned for minimize compliance. The designs returned are are relatively simple solid structures with a low compliance.
17

---

C.4 Starting from an existing design
epoch:
epoch: 25
epoch: 50
epoch: 75
epach: 100
The eiffel tower, dark
30.71
33.81
34.38
34.42
epoch:
epoch:
23-33 v: 35.86
19 vs 37.57
cank bleck outline"
66
35.60
35.64
epoch:
epoch: 25
epach: 50
epach:
apach: 100
rabot care black
ci 5.
Ci 2.8e+09
12
36
26.33
epoch:
epoch:
epoch:
epoch:
epoch:
arch
26.17
28.20
28.44
28.38
30.30
Figure 12: Starting from an existing design. An image generated by Stable Diffusion is converted to a single channel density representation and passed as the starting design encoding for tower design problem. TIDES updates the material layout to resist the force applied at the top center of the design space while preserving the visual features given by text prompt and starting image.
In addition generating design from scratch, TIDES can be used to update existing design satisfy physical constraints. In Figure 12, we present the generation process starting from an existing design. Here, Stable Diffusion [39] is first used generate image from text prompt. This image is then converted to a one channel grey scale image, resized to match the design space, and passed as the starting parameter for TIDES's design encoding. The "eiffel tower" and "st louis arch" starting designs contain floating material and span the entire design space. Over 100 epochs TIDES places material to improve compliance while maintaining visual features conveyed in the text prompt and starting design. The "eiffel tower" design maintains its iconic shape and the thickness of the support struts are increased by TIDES improving resistance to the applied force. The louis arch" design maintains the arch shape with an additional horizontal strut emerging during generation.
The "space needle" and "robot" starting designs represent a more challenging initial condition where, in addition to floating material, the designs are not fully connected and do not span the entire design space. The saucer in the "space needle" is floating above the tower base and the "robot" does not reach the top of the design space. In both cases, the starting designs do not resist the force applied at the top center and return very high compliance values. Over successive epochs, TIDES improves the design by removing floating material, connecting disconnected structures, and placing material to fully cover the design space and resist the applied force. The "space needle" preserves the visually distinctive feature of saucer/orb supported by pronged tower base. TIDES improves compliance during generation by increasing the thickness of the supports and reducing the size of the saucer. The "robot" maintains the rectangular head resting
support body. The arm features do not offer much support and are removed by TIDES during generation with material reallocated and added to thicken the head and body to improve resistance to the applied force. Across all designs, the compliance mask rapidly removes the floating material and the hill function pushes a binary material distribution removing the grey background.
18

---

C.5
Suspended bridge designs for all 30 trials
In this section we include high resolution plots every design returned from 30 trials of the suspended bridge design problem. Each design corresponds to point plotted in Figure 4 To better view the detailed fine features present in each design we recommend zooming in on the individual design.
c: 7.44
c: 7.37
7.36
7.34
c: 7.32
c: 7.30
c: 7.29
7.26
c: 7.21
c: 7.19
7.19
7.17
7.17
7.15
7.14
<: 7.13
c: 7.12
7.06
7.03
c: 7.03
7.02
7.00
6.97
6.90
6.88
c: 6.85
<: 6.83
6.73
c: 6.69
c: 6.61
Figure 13: Designs generated from runs physics loss alone for the suspended bridge design problem with randomized starting canvas.
19

---

c:
9.09
V: 33.06
v: 31.13
V: 33.37
Vt 34.40
V. 33.15
8.66
c: 8.53
8.45
8.44
8.43
33.08
V: 33.13
V:
31.96
32.98
8.36
c: 8.31
c: 8.31
8.30
c: 8.30
31.86
V: 32.93
32.37
33.01
33.33
8.28
8.26
8.24
8.14
8.14
32.64
V: 33.76
V: 31.86
32.45
Ve 33.64
<: 8.13
c: 8.10
c: 8.05
8.05
c: 8.02
32.59
33.25
32.84
33.45
33.25
c:
v: 33.20
V: 33.35
V. 33.91
v: 29.96
V: 33.37
Figure 14: Designs generated from 30 runs of TIDES for the suspended bridge design problem given text prompt "a large arch, dark black outline"..
20

---

C:
c:
9.17
9.15
V. 30.52
V: 31.10
V: 30.44
30.71
V: 28.88
8.92
8.91
8.83
8.78
8.74
30.27
v. 31.59
V: 31.05
32.10
<: 8.74
c: 8.67
c: 8.58
8.56
c: 8.55
31.13
V: 29.83
29.20
29.35
28.83
8.50
8.46
8.44
8.43
8.42 V: 30.47
v: 30.79
V: 31.52
Vc 30.59
29.61
c: 8.34
c: 8.25
c: 8.23
8.23
c: 8.04
29.86
30.00
29.69
28.88
29.20
c:
v: 31.05
V: 31.18
V. 30.93
vs 30.32
V: 30.10
Figure 15: Designs generated from 30 runs of TIDES for the suspended bridge design problem given text prompt "a large hexagon, dark black outline"..
21

---

9.73
c: 9.32
9.12
V: 31.40
v: 32.30
V: 30.47
v: 31.13
V: 29.64
8.97
C: 8.96
8.95
8.87
8.85
31.40
V: 30.96
V: 31.54
30.91
31.32
<: 8.67
c: 8.60
c: 8.53
8.52
c: 8.50
30.79
V: 31.10
31.13
30.81
31.59
8.43
8.42
8.41
8.37
8.29
30.37
V: 31.35
Vc 31.64
31.05
31.35
c: 8.27
c: 8.26
c: 8.23
8.00
c: 8.00
30.62
30.59
32.01
31.54
31.20
c:
v: 31.08
V: 31.03
Vc 30.86
v: 30.15
V: 30.86
Figure 16: Designs generated from 30 runs of TIDES for the suspended bridge design problem given text prompt "a large triangle, dark black outline"..
22

---

C.6 Hoop designs for all 30 trials
In this section include high resolution plots of every design returned from 30 trials of the hoop design problem. Each design corresponds to point plotted in Figure4B, To better view the detailed fine features present each design we recommend zooming on the individual design.
c: 1.17
c: 1.08
1.04
C: 0.98
c: 0.97
c: 0.95
c: 0.94
0.94
c: 0.92
c: 0.92
c: 0.92
c: 0.91
0.91
c: 0.91
<: 0.89
0.88
0.87
0.87
0.86
0.86
0.85
c: 0.84
0.83
0.83
0.82
0.80
0.80
0.79
0.78
0.78
Figure 17: Designs generated from runs physies loss alone for the hoop design problem with randomized starting canvas.
23

---

C: 1.52
C: 1.50
c: 1.46
C: 1.46
1.44
30.62
30.37
31.49
31.03
31.74
a: 144
1.35
c: 1.31
c: 1.27
c: 1.26
31.23
32.98
30.37
32.35
30.62
c: 1.25
c: 1.23
c: 1.22
c: 1.22
1.20
32.23
32.86
32,08
32.62
30,81
c: 1.20
1.18
c: 1.18
c: 1.15
1.15
31.30
32.81
31.69
31.01
31.79
c: 1.07
1.07
c: 1.06
c: 1.04
1.03
30.64
31.30
32.40
31.93
32.74
<: 1.00
0.99
0.97
C: 0.96 31.69
0.93
31.98
Figure 18: Designs generated from 30 runs of TIDES for the hoop design problem given the text prompt "a large arch, dark black outline"..
24

---

C: 1.92
C: 1.76
C: 1.76
C: 1.74
1,70
28.15
29.66
27.37
29.27
30.37
c: 1.59
c: 1.56
c: 1.54
c: 1.52
c: 1.52
28.47
28.42
27.86
28.44
27.42
c: 1.51
c: 1.50
c: 1.44
c: 1.42
1.41
27.69
27.47
26.32
28.12
28.37
c: 1.41
1.40
c: 1.40
c: 1.38
1.36
29.08
27.83
27.76
30.03
30.40
c: 1.36
1.36
c: 1.32
c: 1.30
1.27
28.93
30.62
27.69
29.96
c: 1.27
1.26
1.24
c: 1.21 29.42
1.20
28.15
Figure 19; Designs generated from 30 runs of TIDES for the hoop design problem given the text prompt "a large hexagon, dark black outline"..
25

---

C: 1.63
C: 1.58
C: 1,53
C: 1.53
1.52
28.27
30.05
30.37
30.88
30.27
<: 1.51
c: 1.51
c: 1.43
c: 1.41
1.41
31.64
30.22
30.49
29.86
31.71
c: 1.40
c: 1.39
c: 1.35
C: 1.35
1.29
30.20
31.32
31.84
29.83
29.98
c: 1.29
1.27
c: 1.26
c: 1.26
1.23
29.93
29.00
30.03
31.47
30.27
6: 1.23
1.22
c: 1.22
c: 1.19
1.16
29.81
31.69
29.96
30.03
29.15
<: 1.12
1.09
1.04
1.02
0.97
30.37
31.35
Figure 20: Designs generated from 30 runs of TIDES for the hoop design problem given the text prompt "a large triangle, dark black outline".
26

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
The main claim made in the abstract and introduction is the development framework for generating physically sound designs from text and set of physical constraints. The claim is supported by experimental results presented in Section 4 where designs generated by the approach resist physical deformation in simulation and in the lab while displaying visual support structures and features that align with the text prompt.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We discuss the limitations of the work performed in the main body of the text and include additional discussion in the Appendix A
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
27

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA].
Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide a detailed full and clear description of the TIDES algorithm and
architecture depicted in Figure 2 in Section 3 along with discussion of all settings for each individual run and trial and experimental setup in Appendix We will make our code available.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to model. In general. releasing code and data often one good way to accomplish this, but reproducibilityy can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibiliity In the case of closed-source models, it may be that access to the model is limited in
28

---

some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We describe the experimental configurations Appendix and we will make our code available.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nnis..o cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https https///ooSSuubisse..o
/ //nipscco wwtppppcccgguuicecccccccocccccce
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: In Appendix we provide descriptions of all experimental setups and the hyperparameters used for each run.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We conducted multiple trails and report distribution plots with confidence ellipses and min max range bars where appropriate. In Figure 4 we plot the distribution of performance across 30 trials per prompt along with their confidence ellipse and the min, median, and max values observed for the baseline. In Figure 6 we plot bars that indicate the min and max force observed each displacement across the replicates.
Guidelines:
29

---

The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper does not require extensive computing resources. All runs and trials were performed on a computer with RTX A4000, 32GB Ram, and an Intel i7 processor.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: The authors have reviewed the NeurIPS code of Ethics and the paper conforms, in every respect, with the NeurIPS code of ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
30

---

Justification:
We discuss potential societal impacts in Appendix: A
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
We cite the original papers for all assets used and direct the reader to the original papers for more detailed discussion. We respect the terms of the licenses. All existing assets use permissive free-software licenses e.g. MIT License for CLIP and Apache License 2.0 for the physics simulator.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
31

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperssi thoode com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's ' creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We will make our code available and provide documentation.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the datasct/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing research with human subjects.
Guidelines:
32

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to LLM for what should or should not be described.
33


Reviews:
[{"review_id": "ZiaY8GliMm", "reviewer": "Reviewer_ZiBG", "summary": "The paper introduces TIDES, a method that jointly optimizes structural soundness and text-guided visual features in design. It combines a differentiable FEM simulator + a frozen CLIP model to generate 2.5D designs that are both physically viable and visually aligned but via CLIP guidance. Key contributions include a co-optimization framework, practical tricks like compliance masking and hill-based binarization, and some physical validation via 3D printed tests.", "questions": "- The core idea closely resembles Topology Optimization with Text-Guided Stylization Zhong et al. (2023), which also combines CLIP-based semantics with FEM optimization. Why is this not cited, and how does your method differ meaningfully?\n- At higher resolutions (e.g. 512×512), designs become delicate. Are such fine structures actually manufacturable with  3D printing methods?\n- The paper mentions 3D design, but the method operates on 2D grids with extrusion. Can it support native 3D FEM? If not, the terminology should be clarified.\n- How sensitive is the method to prompt phrasing, e.g. “robot” vs. “bot”? Some discussion on this would clarify the controllability and robustness of your approach.", "limitations": "Yes, the authors include a limitations section (Appendix A) where they acknowledge the 2.5D constraint of their simulator, risks of misuse in generating real-world load-bearing structures, and the potential for bias amplification via CLIP. These points cover both technical and societal concerns sufficiently.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths\n- Quality: The method is technically sounds and includes good design choices like compliance masking and hill-based binarization. Experimental coverage is good and includes 3D-printed validation.\n- Clarity: The paper is well-written with clear figures and thorough ablations that support the claims.\n- Significance: Combines text-conditioned generative models with physics-aware structural design, and relevant in the AI-based computational design space.\n- Originality: While the general idea is not entirely new, the implementation is clean, practical, and backed by real fabrication.\n\nWeaknesses\n- Originality: The core idea (CLIP + FEM optimization) overlaps with prior work (e.g. Topology Optimization with Text‑Guided Stylization, Zhong et al. 2023) and is not acknowledged or compared against.\n- Scope: Designs are 2D with extrusion (~2.5D); while the paper references 3D printing and 3D design domains, the method itself does not support native 3D topology optimization or volumetric simulation\n- Visual control: Visual constraints rely solely on text-to-CLIP embeddings, with no direct geometric control, which limits precision and style control.\n- Limited generalization: The method is currently restricted to single-material, planar settings without manufacturability-aware constraints.", "was_revised": true, "final_justification": "Given authors replies and esp the point around Zhong et al. (2023), I'd let the PC decide. The rebuttal has been okay, but didn't move the needle for me.", "rebuttal": "Thank you for your review. \n\n**Q1. The core idea closely resembles Topology Optimization with Text-Guided Stylization Zhong et al. (2023), which also combines CLIP-based semantics with FEM optimization. Why is this not cited, and how does your method differ meaningfully?**\n\nThe first version of the manuscript under review predates Zhong et al. (2023) and we were previously unaware of this work. We cannot provide additional details given the double-blind review policy but are happy to provide verification of this to the Area or Program Chair upon request. The current submission has been updated to address several of the challenges of linking CLIP with a finite element solver. The approach taken in Zhong can overfit to the visual domain and does not scale well to large designs spaces. These issues are not present in our approach we discuss below:\n\nZhong utilizes RGB channels during the design generation process. This causes overfitting to the visual domain by allowing the model to improve the visual loss by painting over sections of the underlying structure rather than by shaping the physical structure itself. E.g. In Zhong Figure 12, the art deco building has off-white colored windows. These windows are not structural features and are painted on over solid material. \n\nOur work does not use RGB values, the focus is on generating a shared binary (material presence/absence, black or white pixel) distribution. The visual loss tends to favor generating artistically pleasant images that utilize shading rather than binary values. To solve this, we introduced a Hill-function into the design encoding to control material distribution and push pixel/density values towards 0 or 1. E.g. In Figure 3, we present designs generated by our approach for an art deco hotel. In contrast to Zhong the windows present in our designs all correspond to structural features (no material is present). In the designs generated by our approach the structural features are physically shaped by the text prompt, there is no RGB outlet for the model to cheat the loss by painting over structural features.\n\nZhong does not scale well to large design spaces and can result in designs with features that do not offer structural support. Zhong utilizes the Connected Component Labeling (CCL) algorithm to remove unconnected features, this is computed at every optimization step. CCL scales poorly in computational time as the size of the design increases. In our work, we introduce a compliance based masking approach to remove unconnected features. As compliance is already computed for structural optimization there is no additional computational overhead required. Further compliance is physics based and allows the removal of features that are not offering direct structural support. In contrast CCL, can lead to connected features that do not offer structural support. E.g. In Zhong Figure 5b, after applying CCL there is a large amount of material that is connected to the main structure but offers no support, this material is hanging and attached by very thin connections that do not lend themselves well to 3D printing. In contrast, for a similar design problem in our work Figure 4, the support struts do not have these issues while displaying the text-informed features. \n\nWe have added a summary of the above to the related works section and a citation to Zhong et al. 2023.\n\n**Q2. At higher resolutions (e.g. 512×512), designs become delicate. Are such fine structures actually manufacturable with 3D printing methods?**\n\nYes, for delicate structures methods such as support bath based 3D printing or UV resin cure printers support printing designs with intricate details at very high resolution.\n\n**Q3. The paper mentions 3D design, but the method operates on 2D grids with extrusion. Can it support native 3D FEM? If not, the terminology should be clarified.**\n\nYes, the Hill function and compliance based masking approach scale readily to 3D. For the visual loss a simple approach could be bounding the design in a cylinder and then sampling 2D slices to generate the images to compute the loss. To avoid confusion around 3D designs in the current work we state in the abstract: \"2D beam designs that are extruded and 3D printed.\" \n\n**Q4. How sensitive is the method to prompt phrasing, e.g. “robot” vs. “bot”? Some discussion on this would clarify the controllability and robustness of your approach.**\n\nIn practice, a user can provide any text prompt. In this work, we selected diverse prompts covering a wide array of topics to indicate robustness and range from the hello world of image generation \"avacado arm chair\" (Figure 3) to structural shapes such as \"hexagon\" and \"arches\" (Figure 4) to artistic concepts such as \"a giant sisphus pushing a stone boulder\" (Figure 10) and many more. For text prompts that cover very similar subjects e.g. “robot” vs. “bot”, the resulting design will be very similar. We have conducted a run for the subjects \"robot\" and \"bot\" the resulting designs display very similar robotic/bot features. Prompt engineering for CLIP has been well explored in the image generation literature, see section 2.2, and was not a focus of this paper."}, {"review_id": "hlX0iaTiQk", "reviewer": "Reviewer_75ov", "summary": "This paper presents a structural optimization (topology optimization) technique for using text to add control of visual appearance. This is done through the addition of an augmented clip similarity loss and a compliance mask to remove disconnected components from a somewhat standard differentiable simulator based approach to topology optimization over a 2D pixel grid. Mapping of structural grids to the CLIP domain is achieved on the image side by interpreting density grids as greyscale RGB, and on the text side by biasing the text inputs towards black silhouettes by appending phrases such as \"dark, black solid outline\"", "questions": "- What, if any, tuning of parameters and CLIP text prompts (specifically the appended text like \"black\" and \"silhouette\") was needed to get these results, at what granularity was this done, and what were the final parameters used for the experiments? E.g., did the loss weights vary by image size, or by text target, or was there a universal set of values that worked well across experiments, and how big of an impact do these parameters have? Are there cases where one would want to use these weights to meaningfully control the designs?\n- Please give technical details on how the compliance masking works sufficient to fully reproduce the experiments.\n- How well is the density constraint achieved across the test examples, especially compared to the physics only baseline? Is there a useful tradeoff between aesthetic considerations and constraint satisfaction?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- A simple, straightforward, and clean architecture that (except for a few details) should be easy to reproduce\n- Qualitative results look very good -- well aligned to input text prompts.\n\nWeaknesses:\n- Does not give details on the tuning necessary to achieve these results. Specifically, which weight parameters in the optimization loss were used, if these were consistent across trials or tuned for each one, and how much effort was used to do so. Similarly, was tuning (and possibly per-example tuning) required for the appended text strings to bias CLIP towards silhouette?\n- Does not give details on how the compliance masking works\n- Does not give any evaluation on how close to the target density design get (in absolute terms and compared to physics baseline). Since this is the only physics constraint (a soft constraint on density) that is explored, it is important to know how well this technique can balance constraints and optimization goals relative to a setup without the additional vision goal.", "was_revised": true, "final_justification": "The rebuttal addressed my primary evaluation concern and provided a concise but sufficient description of the compliance masking details. I intend to maintain my recommendation.\n\nOther reviewers have raised concerns over applicability to 3D and potential over-claming \"physical soundness\". I agree with the author's that generalization into \"full 3D\" is not claimed nor is necessary for publication (2/2.5D TO is useful in its own right). \"Physically Sound\" is potentially a very strong claim, but the paper does not make a firm definition of what it considers sound other than experimentally the design were 3D print-able. I agree that some of this language could be toned-down, or a brief explanation of what is meant by \"soundness\" added, but think this is not a major enough concern to reject the paper.", "rebuttal": "Thank you for your review. \n\n**Q1. What, if any, tuning of parameters and CLIP text prompts (specifically the appended text like \"black\" and \"silhouette\") was needed to get these results, at what granularity was this done**\n\nWe follow a fairly common prompting strategy of splitting the prompt into two parts the subject followed by the modifier, i.e. \"subject, modifier\" => \"Eiffel tower, dark black outline\". The subjects were selected to cover a wide array of topics to indicate robustness and range from the hello world of image generation \"avacado arm chair\" (Figure 3) to structural shapes such as \"hexagon\" (Figure 4) to artistic concepts such as \"a giant sisphus pushing a stone boulder\" (Figure 10) and many more. The modifier was selected to prompt in the binary region of the images CLIP was trained on. We arrived at the modifier of \"dark black outline\" as all designs should be solid dark black, not have shading and should have sufficient detail to capture the features. As outlines e.g. coloring books, are binary black and white and are detailed representations this was chosen as the modifier. In practice a user can supply any text prompt and as visual appearance is a goal there is some subjectivity on the user level on the modifier selection.\n\n**what were the final parameters used for the experiments?** \n\nIn Appendix B, we provide all parameters used for each experiment. \n\n**did the loss weights vary by image size, or by text target, or was there a universal set of values that worked well across experiments, and how big of an impact do these parameters have?**\n\nThe loss weights did not vary by image size or target text prompt. In Figure 3, the designs at varying resolution all utilize the same parameters. In Figures (1,2,4,5) the designs generated from different text prompts all use the same parameters. All results in the main body of the paper use the same parameters. The visual loss weight of 100 used in the main body of the paper was selected as it is fairly common in the literature to weight the clip score by 100. In Appendix Figure 10 for some of the design problems we increased the visual loss based to balance out an increase in the initial physics loss in some of the more complex design problems. \n\n**Are there cases where one would want to use these weights to meaningfully control the designs?**\n\nYes, if aesthetic performance is valued over physical performance one could increase the visual weight to trade off physical performance. In the current work, our focus was on generating physically sound designs that utilized features specified by text and we opted for weights that preserved physical performance while utilizing visually recognizable features.\n\n**Q2. Please give technical details on how the compliance masking works sufficient to fully reproduce the experiments.**\n\nThe compliance mask is generated by thresholding the element wise compliance returned from the physics simulator i.e. mask = log(compliance) >= threshold; threshold = -20. This returns a mask of zeros and ones where zero indicates a given pixel does not contribute structurally and one indicates it does. The mask is applied by multiplying by the design encoding (Image = mask*design), this is the first image augmentation performed. This ensures the CLIP loss only reflects pixels that are structurally contributing and prevents overfitting to the visual domain (i.e. prevents cheating the clip loss through creating floating features or non-structurally contributing features).    \n\n**Q3. How well is the density constraint achieved across the test examples, especially compared to the physics only baseline?**\n\nThe density constraint is achieved across all examples. Below we have included the mean and standard deviation in density across 30 trials for two different design problems and text prompts.\n\n| text prompt | mean  |   std   \n| -------- | -------- | ------- |\n| a large $\\mathbf{arch}$, dark black solid outline     |  0.301155    |  0.001447\n| a large $\\mathbf{hexagon}$, dark black solid outline  |  0.300743    |  0.001627\n| a large $\\mathbf{triangle}$, dark black solid outline |  0.301071    |  0.001910\n| baseline                                              |  0.301160    |  0.001846\n[suspended bridge design problem]\n\n\n\n| text prompt | mean  |   std   \n| -------- | -------- | ------- |\n| a large $\\mathbf{arch}$, dark black solid outline     |  0.300756      |   0.002440\n| a large $\\mathbf{hexagon}$, dark black solid outline  |  0.300305      |   0.002358\n| a large $\\mathbf{triangle}$, dark black solid outline |  0.299975      |   0.002504\n| baseline                                              |  0.299889      |   0.001756\n[hoop bridge design problem]"}, {"review_id": "79y9yypTuE", "reviewer": "Reviewer_nNGQ", "summary": "The paper presents TIDES, a differentiable pipeline that jointly optimizes a CLIP-based visual loss with structural compliance and material-cost losses so that 2D density fields evolved from a prompt end up both visually faithful and mechanically stiff. Overall, the study demonstrates that adding differentiable physics to a pre-trained generative model can produce manufacturable, prompt-conditioned structures.", "questions": "Please refer to the \"Weaknesses\" section.", "limitations": "yes", "rating": 2, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**:\n* The paper explicitly co-optimize a differentiable finite-element compliance loss (modified SIMP) and a CLIP text-image similarity loss in a single end-to-end loop, which unifies topology optimisation and text-to-image generation.\n* This article provides a detailed explanation of the hyperparameters and visualization results of the experiment, and it has strong reproducibility. Meanwhile, conduct a thorough analysis of the potential limitations and impacts.\n\n**Weaknesses**:\n* All 3D results are 2D densities simply extruded,  so claims of “physically sound designs” are somewhat overstated for real engineering practice.\n* Comparisons are limited (limited comparison methods and datasets/tasks). The paper omits stronger co-design baselines such as recent diffusion-based inverse design methods, making it hard to judge relative performance gains.\n* The manuscript’s novelty derives mainly from combining these known techniques in one pipeline (most core components—SIMP compliance optimization, CLIP-based text supervision and Hill-type density projection are adopted from earlier papers.), not from introducing new algorithms or theory, which is more of a incremental. It would be best to explain what the methodological contributions are.", "was_revised": true, "final_justification": "The rebuttal clarifies the method’s novelty and addresses several technical questions. However, important related works on physics-constrained generation are not sufficiently discussed, and comparisons are missing. Furthermore, evaluation on existing datasets such as DrivAerNet++ could better demonstrate generality, especially for 3D domains with stricter physical constraints. While the contributions are interesting, these gaps prevent a higher score. I will maintain my current score.", "rebuttal": "Thank you for your review.\n\n**W1. All 3D results are 2D densities simply extruded, so claims of “physically sound designs” are somewhat overstated for real engineering practice.**\n\nWe make no claims regarding real engineering practice, the field is not there yet for text based design. Structural optimization is widely used to design physical structures and TIDES generated designs approach the upper performance bound of designs generated by structural optimization, further we take the additional step of printing physical versions of the designs and validating the physical performance under load in the lab.\n\n**W2. Comparisons are limited (limited comparison methods and datasets/tasks). The paper omits stronger co-design baselines such as recent diffusion-based inverse design methods, making it hard to judge relative performance gains.**\n\nAs far as we are aware, there are no existing baselines that utilize text to guide the generation process of a design that resists physical deformation from an applied force. There are no datasets involved as we are using a differentiable physics simulator and a pre-trained text-image model to compute the loss terms. \n\n**W3. The manuscript’s novelty derives mainly from combining these known techniques in one pipeline (most core components—SIMP compliance optimization, CLIP-based text supervision and Hill-type density projection are adopted from earlier papers.)**\n\nThe NeurIPS review guidlines state: \"Does this work offer a novel combination of existing techniques, and is the reasoning behind this combination well-articulated?\"\n\nThis paper is the first to bridge two separate fields of research (text-image generation, and structural optimization). Its findings indicate TIDES, is well-suited for generating designs that resist structural deformation. This addresses a limitation of prior text generation works that are incapable of considering physical constraints and is an innovation toward realizing generated designs in life.\n\n**Hill-type density projection are adopted from earlier papers.**\n\nThe Hill equation (equation 4) comes from bio-chemistry and has not to our knowledge been previously used in the context of structural optimization density projections. In standard structural optimization compliance alone is enough to drive the binary material distribution no extra steps are needed. The introduction of the visual loss leads to a competing design objective with shading encouraging a gray-scale distribution. We solve this by developing a Hill function with a steep slope to push binarization. \n\n**W2. It would be best to explain what the methodological contributions are**\n\nWe developed a Hill function inspired activation (eq. 4) to restrict and push density values to 0 or 1, a compliance-based masking approach to ensure that the visual features offer structural support, a resampling approach to scale between different design resolutions, and a weighted loss to balance competing objectives. The key insight of the paper is that bridging a differentiable physics simulator with a pretrained text-image model allows text informed generation of strucutrally sound designs. This has implications for both the text to design and structural optimization.\n\nFrom the text to design side: The success with using a pretrained text-image model with a differentiable physics simulator indicates that it is not necessary to train a new text to design model from scratch on physics data to generate sound designs. Instead, bridging differentiable physics with pretrained text image models offers a promising new research direction towards realizing generated design in life.\n\nFrom the engineering/structural optimization side, we have shown that a visual text-image loss can be used to convey complex design goals and influence the features in the design while still respecting the underlying physical constraints. This is a promising new direction towards co-design, with text providing an intuitive means for conveying complex design goals to a physics-based optimizer and allowing for diverse design generation guided by text."}, {"review_id": "7cF9fX8sY3", "reviewer": "Reviewer_UJvm", "summary": "This paper presents a generative framework that can synthesize physically sound structural designs according to text instructions. Given a design instance, it leverages a physics simulator to compute deformation compliance and material cost, and uses CLIP to obtain the similarity between the visual appearance and the text guidance. Then, it jointly minimizes a compliance loss, a material cost, and an image-text alignment loss to achieve a functional design that can withstand applied forces. Experiments on several design problems, as well as a 3D-printing evaluation, demonstrate the effectiveness of the proposed method.", "questions": "1.\tThe use of differentiable physics to enhance the physical soundness of generated assets has been explored in several existing methods [a-c]. What distinguishes the proposed method from these works?\n\n2.\tThe paper starts with a design encoding value of 1. Is it possible to initialize the proposed method using existing designs (e.g., from artists) that may not initially withstand the applied force, and then optimize these designs with minimal modifications to ensure they can support the force? This can improve the practical usage of the method.\n\n3.\tThe paper employs varying values for $\\beta_1$ and $\\beta_2$ across different design problems (see Section B.5), suggesting that the proposed method may be sensitive to the selection of hyperparameters. How are these hyperparameters chosen? Did the authors conduct experiments to determine a set of hyperparameters that perform well across all the considered problems?\n\n4.\tThe experiments only compare the proposed method with its own ablations (w/o visual loss or w/o physics loss). How does the method compare with state-of-the-art text-to-image models like Stable Diffusion[d]?\n\n5.\tWhat is the running time for generating a single design?\n\n\nReferences\n\n[a] Physically Compatible 3D Object Modeling from a Single Image. NeurIPS 2024.\n\n[b] Atlas3D: Physically Constrained Self-Supporting Text-to-3D for Simulation and Fabrication. NeurIPS 2024.\n\n[c] DSO: Aligning 3D Generators with Simulation Feedback for Physical Soundness. arXiv 2025.\n\n[d] High-Resolution Image Synthesis with Latent Diffusion Models. CVPR 2022.", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\nThis paper presents an end-to-end framework for generating images of structures from text while ensuring physical accuracy. It leverages the strengths of a multi-modal language model, CLIP, to enhance the alignment between text and images. Additionally, it incorporates a differentiable physics simulator to optimize the structural design for physical soundness. The overall concept is straightforward and effective, as demonstrated by real-world 3D printing experiments.\n\n**Weaknesses**\n\nThere is ample room for improvement in the paper's structure and organization. Specifically, it lacks clear sections and many parts are mixed together, making it difficult for readers to understand. For example, in Section 2, background and related work are intermixed, making it difficult to find an introduction to similar existing works on the same task. Similar issues can also be found for Section 3, and it is suggested to use subsections to organize the paper better.\n\nAdditionally, there are concerns regarding the technical novelty and completeness of the evaluation of the proposed method; see the questions below for details.", "was_revised": true, "final_justification": "The authors' response addresses most of my initial concerns, particularly regarding the distinction from existing work. The one remaining is the experiment using existing designs. Based on the authors' response, I decided to adjust my recommendation to 4.", "rebuttal": "Thank you for your review. \n\n**W1 For example, in Section 2, background and related work are intermixed, making it difficult to find an introduction to similar existing works on the same task.**\n\nIn Section 2 we provide an overview of two independent design problems structural optimization (2.1 similar existing work from the physics side) and  text-image generation with CLIP (2.2 similar existing work for the text-image side).  In Section 3 we make the case there is value in combing these two design problems into a new task. The field is rapidly progressing and at the time of writing there were no existing works on embedding physical constraints into text-image models. The reviewer has brought to our attention very recent concurrent works [a-c] which use a physic loss to tackle a different design problem (stability and image guided design). We have updated section 2 to added a third subsection titled \"physics constrained generative design\" to our related work section based on our response to question 1. \n\n**Q1. The use of differentiable physics to enhance the physical soundness of generated assets has been explored in several existing methods [a-c]. What distinguishes the proposed method from these works?**\n\nThe first version of the manuscript under review predates [a-c]. The current submission has been updated to address several of the challenges of linking CLIP with a finite element solver. Below we discuss how our work differs from [a-c]. \n\n[a] Requires an input image to guide the visual geometry of the physical object returned. Our work differs in that it does not require an input image, we start from a solid block of material and use  a differentiable physics simulator, material cost, and visual loss given by a text prompt to shape the design's structure during optimization. In addition our work is also generative, in that in repeated trials for the same physical constraints and text prompt, our approach generates a distribution of diverse designs e.g. Appendix C.4, C.5 that utilize the text informed feature to satisfy the physical constraint through various support strategies rather than adhering to a specific predefined visual geometry. \n\n[b] and [c] are concerned with generating stable designs, designs that remain upright. In [b] a physics-based loss is formulated for standability and used as regularizer to refine designs for stability. In [c] a dataset of 3D objects with stability scores is generated and used to fine tune a generator. The designs in [b] and [c] rest on a continuous flat or curved surfaces.\n\nOur work differs from these methods in that our focus is on a different design problem, structural optimization. In structural optimization, the goal is to generate a design that resists deformation from an applied force through minimizing compliance. The forces applied in structural optimization can vary in intensity and placement throughout the design space rather than a constant gravitational force. For example, in Appendix Figure 10 we present designs generated from a variety of force applications including: horizontal force application (Dam), multiple point force application (multistory building), varied force application (staircase). Further, the placement of supports is not limited to continuous flat or curved surface e.g. the suspension bridge (Figure 4) or the cantilever beam two-point design (Appendix Figure 10), are suspended and sections of the design do not rest on the ground.\n\n**Q2. The paper starts with a design encoding value of 1. Is it possible to initialize the proposed method using existing designs (e.g., from artists) that may not initially withstand the applied force, and then optimize these designs with minimal modifications to ensure they can support the force? This can improve the practical usage of the method.**\n\nYes, as the design encoding is a direct encoding an existing design could be used as the starting point for optimization. To use an existing designs, all one would have to do is resize the existing design to the same size as the desired design space and pass this as the initial state for the design encoding. We opted to use a starting value of 1 as it offers the nice intuitive interpretation of starting from a solid block of material. For best performance with an existing design we would recommend computing the ratio of black/white pixels in the existing design and setting the material cost higher than this value. \n\n**Q3. The paper employs varying values for and across different design problems (see Section B.5), suggesting that the proposed method may be sensitive to the selection of hyperparameters. How are these hyperparameters chosen? Did the authors conduct experiments to determine a set of hyperparameters that perform well across all the considered problems?**\n\nAll results in the main body of the paper use the same parameters (B.1-B.4). The visual loss weight of 100 was selected as it is fairly common in the literature to weight the clip score by 100. In Appendix Figure 10 we increased the visual loss and adjusted the material cost to balance out an increase in the initial physics loss in some of the more complex design problems. Here there is still overlap in parameters between design problems, 5 of the 6 problems use the same visual weight, 3 use the same material weight of and another 2 use the same weight of 50 the same value used for (B.1-B.4). The method is not particularly sensitive to hyperparameter selection and given the design algorithm is fairly quick and the subjective nature of the visual design goal we found in practice a user can quickly iterate over a couple of parameters and select the one they prefer. We have updated section B.5 to clarify the parameter selection.\n\n\n**Q4. The experiments only compare the proposed method with its own ablations (w/o visual loss or w/o physics loss). How does the method compare with state-of-the-art text-to-image models like Stable Diffusion[d]?**\n\nState-of-the-art text-to-image models like Stable Diffusion cannot be directly compared with the proposed method as they operate over the visual domain (3 channel RGB images), do not utilize density constraints or physical constraints and are unable to generate physically meaningful designs. The w/o physics loss designs we compare against can be evaluated with the physics simulator as they still operate over 1-channel, satisfy the density constraint, and are pushed by our encoding to a binary pixel distribution. \n\n**Q5. What is the running time for generating a single design**\n\nThe slow computational step is the physics simulator on the CPU. The visual loss runs almost instantaneously on the GPU. Generation times varied from less than a minute for low resolution designs and up to ~15 minutes for 512 x 512, a more efficient differentiable physics simulator could greatly speed this up."}, {"review_id": "SvUhnKLN64", "reviewer": "Reviewer_fywP", "summary": "The paper proposed a pipeline for creating 2D designs that follows both the text description and physical constraints. It jointly optimizes a 2-D density grid with differentiable FEM simulation and a frozen CLIP similarity loss, so each design is both mechanically stiff and visually aligned with a text prompt. Experiments show it outperforms physics-only or vision-only baselines on the stiffness–CLIP-score trade-off and remains robust when 3D-printed and mechanically tested.", "questions": "1. Show a minimal 3D example—e.g., plugging your differentiable FEM module into a DreamFusion-style pipeline and optimizing a 3D structure?  This would directly demonstrate that the method generalises beyond 2D.\n2. SDS-Loss Comparison: Replace the CLIP loss with the SDS loss and see if it provides better results.\n3. Provide analysis on hyperparameter sensitivity. How hard is it for a user to find suitable parameters? This will reveal how robust the framework is and offer guidance to practitioners.\n4. Timing and Scaling: Report the wall-clock time per iteration and total training time for each resolution (64 → 512), and compare with other solutions on the same hardware.", "limitations": "yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "# Strengths\n1. First to insert a frozen CLIP semantic loss into differentiable topology optimization, enabling joint vision-physics gradients.\n2. Well-designed experiments with strong physics-only / vision-only baselines and 3D-printed real-world mechanical validation.\n3. Offers a text-driven yet mechanically sound design pipeline, potentially impactful for architecture and mechanical engineering.\n4. Pipeline diagram and loss equations are concise and easy to follow.\n\n# Weaknesses\n1. Limited to 2D structures; no convincing argument or demo for 3D extension.\n2. Many tunable hyperparameters, but no systematic sensitivity study.\n3. Alternative Design: Diffusion-based losses (e.g., SDS) could replace CLIP loss\n4. Runtime: The paper states only the hardware used but omits per-iteration or total wall-clock times, making efficiency hard to judge.", "was_revised": true, "final_justification": "Thank the authors for their rebuttal. My concern for Q3 is resolved, but the rest are not:\n\nQ1, extruding a 2D design to 3D is not real 3D. Would love to see results with more spatial constraints.\nQ2 is rather easy to implement. It would be better to be included in this paper's revision rather than as future work.\nQ4, please include a more detailed timing for specific settings.\n\nTherefore, I decided to keep my current rating.", "rebuttal": "Thank you for your review.\n\n**W2. Many tunable hyperparameters, but no systematic sensitivity study.**\n\nThere are 2 tunable hyperparameters: $\\beta_1$ and $\\beta_2$, the weight on the material cost and visual loss. The selection of these values has a subjective component as the visual appearance is a design goal. Given the speed of the algorithm at low resolution and that the parameters scale across resolution (Figure 3 all designs are generated with the same parameters), a user can quickly iterate over designs at low resolution to find the parameters that match their desired appearance and physical performance and then run at the desired resolution. \n\n\n**Q1. Show a minimal 3D example—e.g., plugging your differentiable FEM module into a DreamFusion-style pipeline and optimizing a 3D structure? This would directly demonstrate that the method generalises beyond 2D.**\n\n\nThe three point bending problem for beam design can be simplified as a 2D problem with extrusion as in section 4.3. Here, the beams are designed with our approach, 3D printed through extruding the design, and tested under load for a 3 point-bending problem. The results for the 3D beam align with the performance in simulation validating the approach. A current limitation of this approach as mention in the paper is the simulation environment restricts 3D designs to layering of 2D force applications. \n\n**Q2. SDS-Loss Comparison: Replace the CLIP loss with the SDS loss and see if it provides better results.**\n\nThe work under review presents the concept of linking a differentiable physics simulator and text-image model and demonstrate that this is a promising option for placing physical constraints on a pretrained text-image model. The success of a visual loss in this regard is a finding of this work. In future directions for this work such as placing physical constraints on a diffusion model, SDS could be used. \n\n**Q3. Provide analysis on hyperparameter sensitivity. How hard is it for a user to find suitable parameters? This will reveal how robust the framework is and offer guidance to practitioners.**\n\nAll results in the main body of the text use the same hyperparameters. Here the visual weight was set as 100 as it is fairly common in the literature to weight the clip score by 100. The hyparameters have a subjective component given the visual appearance of the design is a design target. Given the speed of the algorithm at low resolution and that the parameters scale across resolution (Figure 3 all designs are generated with the same parameters), a user can quickly iterate over designs at low resolution to find the parameters that match their desired appearance and physical performance and then run at the desired resolution. \n\n**Q4. Timing and Scaling: Report the wall-clock time per iteration and total training time for each resolution (64 → 512), and compare with other solutions on the same hardware.**\n\nThe slow step is the physics simulation, the visual loss runs almost instantaneously on the GPU. TIDES is slightly slower then physic alone baseline as you need to run both and pass the gradients between the CPU and GPU. Generation times varied from less than a minute for low resolution designs and up to 15 minutes for 512 x 512, a more efficient differentiable physics simulator could greatly speed this up."}]

Author rebuttal:

Meta-review: The paper proposes a method that generates 2D designs by jointly optimizing a text-to-image visual loss (measured through CLIP), a material cost, and a physics-based loss (measured through a differentiable physics simulator). The main strengths of the paper are: (a) its novelty based on combining SIMP compliance optimization, CLIP-based text supervision, and Hill-type density projection to achieve physically sound designs; and (b) a generic framework that can accommodate various 2D forces (beyond gravity, as supported by prior work). There were a few identified weaknesses, such as: (a) the method is limited to 2D structures, (b) many hyperparameters may require hand-tuning for different designs, and (c) similarities to prior work that are not fully discussed, specifically Zhong et al., Topology Optimization with Text-Guided Stylization, Structural and Multidisciplinary Optimization, 2023.

The paper received one accept, one reject (nNGQ), two borderline accepts, and one review (ZiBG) where the reviewer deferred the decision to the "PC". The AC carefully read both the submission and Zhong et al. (2023), and agrees with the authors that the latter can overfit to the visual domain due to its style optimization loss, whereas the proposed method is more technically sound. Reviewer nNGQ argued that the paper does not compare with "important related works on physics-constrained generation". However, the AC concurs with the authors’ view that the works cited address different design problems (e.g., stability under a singular gravitational force). Since no reviewer provided a strong and detailed justification for rejecting the paper solely based on prior work addressing different goals, the AC recommends acceptance.

That said, the authors are strongly encouraged to expand their discussion of all prior and concurrent work. In particular, they are strongly encouraged to include a figure comparing with Zhong et al. (2023) -- such as the art deco building example mentioned in the rebuttal where Zhong et al. fails. More broadly, the authors are encouraged to incorporate all clarifications from the rebuttal and the discussion into the final version of the paper and supplementary material!
Decision: Accept (poster)