SYSTEM AND METHOD FOR VEHICLE DESIGN

Systems, methods, and other embodiments described herein relate to generating vehicle design using a diffusion model. In one embodiment, a method includes generating an image based on a reference image and a text description of the reference image using a diffusion model. The method includes, based on whether the image meets a predetermined constraint, generating an intermediate image based on the image and the predetermined constraint using an analysis model such that the intermediate image is incrementally closer to meeting the predetermined constraint than the image, regenerating the image based on the intermediate image using the diffusion model, and outputting the image.

TECHNICAL FIELD

The subject matter described herein relates, in general, to systems and methods for vehicle design.

BACKGROUND

Machine learning models are useful in generating data. Machine learning models may generate data based on text. However, the context in which the machine learning models can use the text is limited. As such, the machine learning models are incapable of generating data that meet a constraint provided in text format.

SUMMARY

In one embodiment, a system for generating vehicle design using a diffusion model is disclosed. The system includes a processor and a memory in communication with the processor. The memory stores machine-readable instructions that, when executed by the processor, cause the processor to generate an image based on a reference image and a text description of the reference image using a diffusion model. The memory stores machine-readable instructions that, when executed by the processor, cause the processor to, based on whether the image meets a predetermined constraint, generate an intermediate image based on the image and the predetermined constraint using an analysis model such that the intermediate image is incrementally closer to meeting the predetermined constraint than the image, regenerate the image based on the intermediate image using the diffusion model, and output the image.

In another embodiment, a method for generating vehicle design using a diffusion model is disclosed. The method includes generating an image based on a reference image and a text description of the reference image using a diffusion model. The method includes, based on whether the image meets a predetermined constraint, generating an intermediate image based on the image and the predetermined constraint using an analysis model such that the intermediate image is incrementally closer to meeting the predetermined constraint than the image, regenerating the image based on the intermediate image using the diffusion model, and outputting the image.

In another embodiment, a non-transitory computer-readable medium for generating vehicle design using a diffusion model is disclosed. The non-transitory computer-readable medium includes instructions that, when executed by a processor, cause the processor to generate an image based on a reference image and a text description of the reference image using a diffusion model. The instructions include instructions to, based on whether the image meets a predetermined constraint, generate an intermediate image based on the image and the predetermined constraint using an analysis model such that the intermediate image is incrementally closer to meeting the predetermined constraint than the image, regenerate the image based on the intermediate image using the diffusion model, and output the image.

DETAILED DESCRIPTION

Systems, methods, and other embodiments associated with systems and methods for vehicle design are disclosed. Utilizing Generative AI (Artificial Intelligence) tools for vehicle design may lead to significant inefficiency as the Generative AI tools are not capable of considering constraints when generating vehicle designs, and as such, may generate vehicle designs that do not meet, as an example, engineering constraints such as drag coefficient. This may lead to generating multiple vehicle design iterations before achieving a vehicle design that meets the desired constraints.

Current methods for generating vehicle design using Generative AI tools may not include breakthrough creativity as the Generative AI tools may generate designs based on a distribution of existing designs. Further and as previously mentioned, the Generative AI tools do not consider constraints when generating designs. Although Generative AI tools may consider generalized text prompts that may further describe a reference image to be inputted to the Generative AI tools, the Generative AI tools cannot consider specific machine-interpretable representations such as drag coefficients and/or vehicle weight distribution when generating a design.

Accordingly, systems, methods, and other embodiments associated with vehicle design are disclosed. As an example, the system receives a reference image (e.g., an image of a vehicle) and a text prompt that describes the reference image (e.g., “Side view of a vehicle”). The system inputs the reference image and the text prompt to a diffusion model. As an example, the diffusion model may be a stable diffusion model. The diffusion model outputs an image based on the reference image and the text prompt. The system inputs the image into an analysis model. The system receives one or more predetermined constraints, such as a drag coefficient value and a vehicle weight distribution ratio. The analysis model determines whether the image meets the predetermined constraints. In the case that the image meets the predetermined constraints, the system outputs the image. In the case that the image does not meet the predetermined constraints, the analysis model alters the image generating an intermediate image that is incrementally closer to meeting the predetermined constraints than the image. As such, although the intermediate image may not meet the predetermined constraints, the intermediate image is closer to meeting the predetermined constraints than the image. The system then feeds the intermediate image into the diffusion model and the diffusion model generates an updated image based on at least the intermediate image. The system then feeds the updated image back into the analysis model to determine whether the image meets the predetermined constraints. The cycle may continue until the analysis model determines that the image meets the predetermined constraints.

The embodiments disclosed herein present various advantages over conventional technologies that generate vehicle design. First, the embodiments increase time efficiency based on the feedback loop between the diffusion model and the analysis model. Second, the embodiments are able to produce and fine-tune novel artistic designs that satisfy engineering constraints. Third, the embodiments alleviate miscommunication and extensive communication iterations between vehicle designers and vehicle engineers, ensuring that vehicle design goals and engineering constraints are met with significantly fewer communication iterations.

FIG.1illustrates a data flow of a vehicle design system100. The vehicle design system100may include various elements, which may be communicatively linked in any suitable form. As an example, the elements may be connected, as shown inFIG.1. Some of the possible elements of the vehicle design system100are shown inFIG.1and will now be described. It will be understood that it is not necessary for the vehicle design system100to have all the elements shown inFIG.1or described herein. The vehicle design system100may have any combination of the various elements shown inFIG.1. Further, the vehicle design system100may have additional elements to those shown inFIG.1. In some arrangements, the vehicle design system100may not include one or more of the elements shown inFIG.1.

The vehicle design system100includes one or more diffusion models110and one or more analysis models120. The diffusion model(s)110and the analysis model(s)120may be any suitable machine learning models.

The diffusion model110is a generative model. Generative models are a class of machine learning models that can generate new data based on training data. As an example, the diffusion model110is capable of generating images such as high-resolution images. The diffusion model110may generate an image140based on a reference image130and a text description135of the reference image130. The text description135provides context to the diffusion model(s)110. As such, the diffusion model110performs a guided image generation where the diffusion model110generates a detailed image140conditioned on both the reference image130and the text description135. As an example, the diffusion model110may be a stable diffusion model, capable of generating and/or modifying images140,150based on text prompts135. In general, the diffusion model110receives a reference image130and a text prompt135. The reference image130may be in any suitable format, such as in two-dimensional format, a three-dimensional format, and/or an audio format. The text prompt135describes the reference image130and is in a text format. In response to receiving the reference image130and the text prompt135, the diffusion model110outputs an image140,150in any suitable format based on and guided by the reference image130and the text prompt135.

The analysis model120is a model that is capable of analyzing an image140and determining based on the analysis, whether a predetermined constraint has been met. The analysis model is also capable of altering the image140, thereby generating an intermediate image145that is incrementally closer to meeting the predetermined constraint than the image140. As such, the analysis model120may also be a generative model.

The predetermined constraint is a quantifiable or measurable characteristic of the object represented by the image140. As an example, in the case of the image140being a vehicle as shown, the predetermined constraint may be a drag coefficient, a manufacturability criterion, a vehicle dimension, a vehicle structural strength, and/or a vehicle weight distribution. More generally, the predetermined constraint can be any constraint that is a differentiable function of the image140. As such, the predetermined constraint can be a constraint that can be derived from the image140. As an example, a differentiable analysis module may process the image and output any parameter of interest.

As an example of the predetermined constraint being a drag coefficient, the analysis model120receives a maximum drag coefficient value. In such an example, the analysis model120may determine the drag coefficient of an image140based on the size and shape of the image140using a suitable formula and/or method. The analysis model120may then compare the drag coefficient of the image140to the maximum drag coefficient value. In a case where the drag coefficient of the image140exceeds the maximum drag coefficient value, the analysis model120may alter the image140to reduce the drag coefficient of the image140. The analysis model may alter the image140by altering the shape of the image140and/or by altering the size of the image140, creating the intermediate image145. The analysis model120may alter the image140without considering whether the resulting intermediate image145is a feasible design. As an example and as shown, to reduce the surface area of the image140, the analysis model120may remove a portion of the front hood and a portion of the back of the vehicle, making the vehicle more streamlined. The analysis model120may determine that with the vehicle being more streamlined, the drag coefficient value is reduced and as such, is incrementally closer to meeting the maximum drag coefficient value.

As another example (not shown), to reduce the surface area of the image140, the analysis model120may remove a portion of a wheel such that the wheel is no longer round. The analysis model120may determine that with less surface area, the drag coefficient value is reduced and as such, is incrementally closer to meeting the maximum drag coefficient value. The analysis model120may output the intermediate image145in any suitable format. As an example, the analysis model120may output the intermediate image145in a two-dimensional format, a three-dimensional format, an audio format, and/or a text format to the diffusion model110.

In a case where the drag coefficient of the image140is less than or equal to the maximum drag coefficient value, the analysis model120may output a signal indicating that the drag coefficient of the image140is less than or meets the maximum drag coefficient value. In response to the signal, the diffusion model110may output the image150.

As an example of the predetermined constraint being a manufacturability criterion, the analysis model120may receive a manufacturability criterion. A manufacturability criterion refers to factors to be considered during the manufacturing process of the object represented by the image140. As an example, a manufacturability criterion may be the time in which the object is to be produced, the amount of computer and/or power resources available to produce the object, and/or the amount of material required to produce the object. The analysis model120may receive one or more manufacturability criteria. The manufacturability criteria may be based on a checklist of various factors that affect the manufacturing process of the object.

The analysis model120may apply any suitable method or formula to determine whether the object represented by the image140can be produced within the provided manufacturability criteria. As an example, the analysis model120may determine the time in which the object represented by the image140would be produced based on the size and shape of the image140. In the case that the object represented by the image140cannot be produced within the provided manufacturability criteria, the analysis model120may alter the image140, as previously mentioned, to generate the intermediate image145that may be closer to meeting the manufacturability criteria. The analysis model120may output the intermediate image145to the diffusion model110. In the case that the object represented by the image140can be produced within the manufacturability criteria, the analysis model120may output a signal indicating that the image140meets the manufacturability criteria. In response to the signal, the diffusion model110may output the image150.

As an example of the predetermined constraint being a vehicle dimension, the analysis model120may receive a minimum or maximum vehicle dimension. The vehicle dimension may be related to the whole vehicle or a portion of the vehicle such as the wheel of the vehicle, the body of the vehicle, the window(s) of the vehicle, the front hood of the vehicle, and/or the trunk of the vehicle. The analysis model120may receive one or more vehicle dimensions. The analysis model120may apply any suitable method or formula to determine whether the object represented by the image140meets the vehicle dimension(s). In the case that the object represented by the image140does not meet the vehicle dimension(s), the analysis model120may alter the image140, as previously mentioned, to generate the intermediate image145that may be closer to meeting the vehicle dimensions. The analysis model120may output the intermediate image145to the diffusion model110. In the case that the object represented by the image140meets the vehicle dimension(s), the analysis model may output a signal indicating that the image meets the vehicle dimension(s). In response to the signal, the diffusion model110may output the image150.

As an example of the predetermined constraint being a vehicle structural strength, the analysis model120may receive any suitable vehicle structural strength values such as a strength-to-weight ratio. The analysis model120may apply any suitable method or formula to determine whether the object represented by the image140meets the vehicle structural strength value(s). In the case that the object represented by the image140does not meet the vehicle structural strength value(s), the analysis model120may alter the image140, as previously mentioned, to generate the intermediate image145that may be closer to meeting the vehicle structural strength value(s). The analysis model120may output the intermediate image145to the diffusion model110. In the case that the object represented by the image140meets the vehicle structural strength value(s), the analysis model120may output a signal indicating that the image meets the vehicle structural strength value(s). In response to the signal, the diffusion model110may output the image150.

As an example of the predetermined constraint being a vehicle weight distribution, the analysis model120may receive a vehicle weight distribution ratio, referring to the distribution of the weight of the vehicle on the wheels of the vehicle. As an example, the vehicle weight distribution ratio may be a ratio of 50-50, such that 50 percent of the weight of the vehicle is on the front wheels and 50 percent of the weight of the vehicle is on the rear wheels. The analysis model120may apply any suitable method or formula to determine whether the object represented by the image140meets the vehicle weight distribution ratio. In the case that the object represented by the image140does not meet the vehicle weight distribution ratio, the analysis model120may alter the image140, as previously mentioned, to generate the intermediate image145that may be closer to meeting the vehicle weight distribution ratio. The analysis model120may output the intermediate image145to the diffusion model110. In the case that the object represented by the image140meets the vehicle weight distribution ratio, the analysis model120may output a signal indicating that the image140meets the vehicle weight distribution ratio. In response to the signal, the diffusion model110may output the image150.

In general, the analysis model120receives an image140, alters the image140to an intermediate image145based on whether the image140meets the predetermined constraint(s) and outputs the intermediate image145in any suitable format such as in a two-dimensional format, a three-dimensional format, and/or an audio format to the diffusion model110.

In one embodiment, the analysis model120may receive more than one predetermined constraint. As such, the analysis model120may receive the predetermined constraint and a second predetermined constraint. The analysis model120may generate an intermediate image145based on the image140and the predetermined constraint, as previously mentioned. In one example, the analysis model120may generate a second intermediate image based on the image140and the second predetermined constraint. In such an example, the analysis model120may generate and output two intermediate images—an intermediate image145and a second intermediate image. In another example, the analysis model120may generate a second intermediate image based on the intermediate image145, the image140, the predetermined constraint, and the second predetermined constraint. In such an example, the analysis model120may generate two intermediate images—an intermediate image145and a second intermediate image and then, output the second intermediate image.

As an example and as shown, the vehicle design system100receives a reference image130of a vehicle and a text prompt135describing the reference image130. More specifically, the diffusion model110receives the reference image130and the text prompt135. The diffusion model110generates an image140based on the reference image130and the text prompt135. The diffusion model110outputs the image140to the analysis model120.

The analysis model120receives the image140and one or more predetermined constraints. The analysis model120determines whether the image140meets the predetermined constraint(s) as previously disclosed. In the case where the analysis model120determines that the image140meets the predetermined constraint(s), the analysis model120may output a signal to the diffusion model110to output the image140. In response to the signal, the diffusion model110may output the image150. In the case where the analysis model120determines that the image140does not meet the predetermined constraint(s), the analysis model120may alter the image140to an intermediate image145such that the intermediate image145is incrementally closer to meeting the predetermined constraint(s) than the image140.

The diffusion model110receives the intermediate image145and generates an image140based on the intermediate image145and the text prompt135. In another embodiment, the diffusion model110generates an image140based on the intermediate image145, the reference image130, and the text prompt135. In another embodiment where the analysis model120generates and outputs two intermediate images (including the intermediate image145), as previously mentioned, the diffusion model110generates an image140based on the two intermediate images (including the intermediate image145) and the text prompt135. Alternatively and/or additionally, the diffusion model110generates an image140based on the two intermediate images (including the intermediate image145), the reference image130, and the text prompt135.

The diffusion model110outputs the image140to the analysis model120and as disclosed above, the analysis model120receives the image140, determines whether the image140meets the predetermined constraint(s), and alters the image140or outputs a signal as previously disclosed. In other words, the cycle of the generation of images140by the diffusion model110and intermediate images145by the analysis model120may continue until the analysis model120determines that the image140meets the predetermined constraint(s).

With reference toFIG.2, one embodiment of the vehicle design system100ofFIG.1is further illustrated. The vehicle design system100is shown as including a processor210. Accordingly, the processor210may be a part of the vehicle design system100, or the vehicle design system100may access the processor210through a data bus or another communication path. In one or more embodiments, the processor210is an application-specific integrated circuit (ASIC) that is configured to implement functions associated with a control module230. In general, the processor210is an electronic processor, such as a microprocessor, that is capable of performing various functions as described herein.

In one embodiment, the vehicle design system100includes a memory220that stores the control module230and/or other modules that may function in support of generating vehicle design. The memory220is a random-access memory (RAM), read-only memory (ROM), a hard disk drive, a flash memory, or another suitable memory for storing the control module230. The control module230is, for example, machine-readable instructions that, when executed by the processor210, cause the processor210to perform the various functions disclosed herein. In further arrangements, the control module230is a logic, integrated circuit, or another device for performing the noted functions that includes the instructions integrated therein.

Furthermore, in one embodiment, the vehicle design system100includes a data store270. The data store270is, in one arrangement, an electronic data structure stored in the memory220or another data store, and that is configured with routines that can be executed by the processor210for analyzing stored data, providing stored data, organizing stored data, and so on. Thus, in one embodiment, the data store270stores data used by the control module230in executing various functions.

For example, as depicted inFIG.2, the data store270includes the images240, the intermediate images245, and the predetermined constraints250, along with, for example, other information that is used and/or produced by the control module230. The images240may include the images140generated by the diffusion model110. The intermediate images245may include the intermediate images145and/or the second intermediate images generated by the analysis model120. The predetermined constraints250may be generated and/or entered using any suitable means such as by a user.

While the vehicle design system100is illustrated as including the various data elements, it should be appreciated that one or more of the illustrated data elements may not be included within the data store270in various implementations and may be included in a data store that is external to the vehicle design system100. In any case, the vehicle design system100stores various data elements in the data store270to support functions of the control module230.

In one embodiment, the control module230includes instructions that, when executed by the processor(s)210, cause the processor(s)210to generate an image140based on a reference image130and a text description135of the reference image130using a diffusion model110.

In one or more arrangements and as mentioned above, the control module230can feed the reference image130and the text description135to the diffusion model110and the diffusion model110can generate the image140.

In one embodiment, the control module230includes instructions that, when executed by the processor(s)210, cause the processor(s)210to determine whether the image140meets a predetermined constraint250using an analysis model120. In one or more arrangements, the control module230can feed the image140to the analysis model120, and as previously disclosed, the analysis model120may determine whether the image140meets the predetermined constraint(s)250.

In one embodiment, the control module230includes instructions that, when executed by the processor(s)210, cause the processor(s)210to, based on whether the image140meets a predetermined constraint250, generate an intermediate image145based on the image140and the predetermined constraint250using the analysis model120, such that the intermediate image145is incrementally closer to meeting the predetermined constraint250than the image140. In one or more arrangements and as mentioned above, the control module230can feed the image140and the predetermined constraint(s)250to the analysis model120and based on whether the image140meets the predetermined constraint(s), the analysis model120may generate the intermediate image145. As previously mentioned, the intermediate image145may be based on the image140and the predetermined constraint(s)250.

In one embodiment, the control module230includes instructions that, when executed by the processor(s)210, cause the processor(s)210to, based on whether the image240meets a second predetermined constraint250, generate a second intermediate image245based on the image240and the second predetermined constraint250using the analysis model120, such that the second intermediate image245is incrementally closer to meeting the second predetermined constraint250than the image240. In one or more arrangements and as mentioned above, the control module230can feed the image240and the predetermined constraints (i.e., the predetermined constraint and the second predetermined constraint)250to the analysis model120and based on whether the image240meets the predetermined constraint(s)250, the analysis model120may generate the intermediate image245and/or the second intermediate image245. As previously mentioned, the second intermediate image245may be based on the image240, the intermediate image245, the predetermined constraint250, and/or the second predetermined constraint250.

In one embodiment, the control module230includes instructions that, when executed by the processor(s)210, cause the processor(s)210to, based on whether the image140meets the predetermined constraint(s)250, regenerate the image140based on the intermediate image(s)245, which may include the intermediate image145and the second intermediate image, using the diffusion model110. In one or more arrangements, the control module230feeds the intermediate image(s)245to the diffusion model110and the diffusion model110regenerates an image140based on the intermediate image(s)245and the text description135.

In one embodiment, the control module230includes instructions that, when executed by the processor(s)210, cause the processor(s)210to, based on whether the image140meets the predetermined constraint(s)250, output the image150. In one or more arrangements, the control module230feeds the image140from the diffusion model110to the analysis model120. In the case that the analysis model120determines that the image140meets the predetermined constraint(s)250, the analysis model120can send a signal to the control module230, indicating that the image140meets the predetermined constraint(s), the control module230may then output the image140generated by the diffusion model110as the output image150. In the case that the analysis model120determines that the image140does not meet the predetermined constraint(s)250, the analysis model120can alter and generate the intermediate images145,245as previously mentioned.

FIG.3is a flowchart illustrating one embodiment of a method300associated with vehicle design. The method300will be described from the viewpoint of the vehicle design system100ofFIGS.1-2. However, the method300may be adapted to be executed in any one of several different situations and not necessarily by the vehicle design system100ofFIGS.1-2.

At step310, the control module230may cause the processor(s)210to generate an image140based on a reference image130and a text description135of the reference image130using a diffusion model110. The control module230may feed the reference image130and the text description135of the reference image130to the diffusion model110, and the diffusion model110may generate an image140based on the reference image130and the text description135of the reference image130. The next step is step320.

At step320, the control module230may cause the processor(s)210to determine whether the image140meets a predetermined constraint250. As an example, the control module230may feed the image140to an analysis model120, and the analysis model120may determine whether the image140meets the predetermined constraint250. In the case where the analysis model120determines that the image140does not meet the predetermined constraint250, the next step is step330. In the case where the analysis model120determines that the image140meets the predetermined constraint250, the next step is step350.

At step330, the control module230may cause the processor(s)210to generate an intermediate image145based on the image140and the predetermined constraint250using the analysis model120such that the intermediate image145is incrementally closer to meeting the predetermined constraint250than the image140. The next step is step340.

At step340, the control module230may cause the processor(s)210to regenerate the image140based on the intermediate image145using the diffusion model110. The next step is step320.