Generative model for inverse design of materials, devices, and structures

A photonic device for splitting optical beams includes an input port configured to receive an input beam having an input power, a power splitter including perturbation segments arranged in a first region and a second region of a guide material having a first refractive index, each segment having a second refractive index, wherein the first region is configured to split the input beam into a first beam and a second beam, wherein and the second region is configured to separately guide the first and second beams, wherein the first refractive index is greater than the second refractive index, and output ports including first and second output ports connected the power splitter to respectively receive and transmit the first and second beams.

FIELD OF THE INVENTION

The invention generally related to method and system for training a device design network for use conditional variational autoencoder to randomly generate materials, device, or structural designs.

BACKGROUND OF THE INVENTION

In many areas of materials, devices, and structures, designing is a challenge because tens, hundreds, or even more parameters need to be optimized simultaneously, and each simulation or experiment to verify the updated characteristics for new parameter sets takes a long time. So efficient optimization methods are desired.

Inverse design of optical devices using deep neural network for regression in forward or inverse directions have been done before. (Tahersima et al., Scientific Reports). Once the inverse model is fully trained, it can theoretically generate the design parameters for us. However, the previous inverse neural network model is for optimizing the binary structure (such as 0 or 1), which reduce the dimension of the actual optimization problem. This may bring up some limitations such as narrower bandwidth and semi optimized result which requires to be further optimized. There is a need to construct a better generative model to be used for more sophisticated optimization problems.

SUMMARY OF THE INVENTION

The invention proposes to use a conditional variational autoencoder, combined with an adversary network to randomly generate device designs wherein desired device performances are given as conditions. Active training (co-training) can be added for further improving the performance.

Some embodiments of the present disclosure are the silicon photonics splitters based on a rectangular or square structure with periodic holes. The dimension of the final square is 2.25 μm by 2.25 μm. There are 400 holes over the square, and these holes have variable hole diameter ranging from 40 μm to 90 μm. The fully trained conditional Variational Autoencoder (CVAE) model can generate different hole vectors combinations based on different splitting ratio inputs. The overall transmission efficiency for all the generated devices is around 90% across a very broad bandwidth (from 1300 nm to 1800 nm), with negligible insertion loss (below −25 dB). Such model can be applied to different devices such as wavelength splitter, mode converter, directional coupler etc.

Some of the embodiments are based on the training data combined with two different datasets: first is the “semi-optimized” results with binary hole size (either no hole or 90 nm hole). The bandwidth for these devices is relatively low (100 nm). The second part of the dataset is combination of multiple patterns along with its performances. According to embodiments, it is shown that with the above “semi-optimized” results, we can train model which is capable of generating devices with excellent performance (90% total transmission) over a broad bandwidth (500 nm). In according to some embodiments of the present disclosure, the model structure is a Conditional Variational Auto Encoder along with the adversarial block, which is based on Bayesian Theorem. It wants the model to underlie the probability distribution of data so that it could sample new data from that distribution. Our training data are generated by doing FDTD simulations. The data are constructed by the following: several DBS simulations and some random generated patterns. The training patterns are all binary holes and the bandwidth is relatively small (1500-1600 nm). The total training data is ˜15,000.

According to some embodiments of the present invention, a system for training a device design network for generating a layout of a device is provided. The system may include an interface configured to acquire input data of a device;a memory to store the device design network including first and second encoders, first and second decoders, and first and second adversarial blocks; and a processor, in connection with the memory, configured to: update the first and second encoders and the first and second decoders based on a first loss function and a third loss function to reduce a difference between the input data and output data of the first and second decoders; and update the first and second adversarial blocks by maximizing a second loss function.

Further, some embodiments of the present invention can provide a computer-implemented training method for training a device design network. In this case, the method comprising steps of: acquire input data of a device via an interface; update first and second encoders and first and second decoders based on a first loss function and a third loss function to reduce a difference between the input data and output data of the first and second decoders; and update first and second adversarial blocks by maximizing a second loss function.

Yet further, some embodiments of the present invention are based on recognition that a computer-implemented method can be provided for generating a layout of a device using a device generating network. The computer-implemented method may include steps of acquiring input data of the device via an interface; feeding the input data into the device generating network, wherein the device generating network is pretrained by a computer-implemented training method, wherein the computer-implemented training method is configured to acquire input data of a device via an interface; update first and second encoders and first and second decoders based on a first loss function and a third loss function to reduce a difference between the input data and output data of the first and second decoders; and update first and second adversarial blocks by maximizing a second loss function. The computer-implemented method further includes generating layout data of the layout of the device using the pretrained device generating network and storing the layout data into a memory.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG.1is the general structure of the system100including a neural network modules trained to provide a layout of a device, according to some embodiments of the present invention. The system100includes an interface115, a processor120, a storage104, a memory106. The storage104includes a device generating module200that includes encoder network modules301and301′, decoder network modules401and401′ and adversarial module (blocks)501and501′. The storage104may include a mapping algorithm108configured to generate a layout of an actual device109, or the mapping algorithm108may be stored into another memory (not shown). The interface115is configured to communicate between the memory106, the storage104, the processor120and the mapping algorithm108. The interface115is also configured to receive input data including a user-desired transmission information101and a Gaussian distribution102via an input device outside the system100. In some cases, the user-desired transmission information101and Gaussian distribution102may be stored into the memory106or the storage104. The desired transmission information (101) and the standard Gaussian distribution (102) are fed to the encoder and decoder neural network modules301,301′,401and401′ via an interface. The neural network modules301,301′,401,401′,500and500′ are pretrained, such that the system100can generate the corresponding hole vector pattern (107) of the device. The system100applies the mapping algorithm (1000), to draw/generate a layout of the actual device. Such network has been verified under a square based splitter model (1100) as one of embodiments of the present invention. Although the following embodiments of the present invention exemplary show the square based splitter model, it should be noted that the shape of the splitter (model) is not limited to a square. For instance, other shapes including rectangular, circular, oval, symmetry shapes, asymmetry shapes or an arbitrary shape that includes any of them may be used. In this case, such a splitter is configured to include an input port configured to receive an input beam having an input power, a power splitter including perturbation segments arranged in a first region and a second region of a guide material having a first refractive index, each segment having a second refractive index, wherein the first region is configured to split the input beam into a first beam and a second beam, wherein and the second region is configured to separately guide the first and second beams, wherein the first refractive index is greater than the second refractive index, and output ports including first and second output ports connected the power splitter to respectively receive and transmit the first and second beams.

Once the neural network modules are fully trained, the system can generate the splitter with any arbitrary splitting ratio the users want instantly. Some results using the system100indicate that those devices have the overall transmission at around 93% across a very broadband, which is really hard (in terms of time and efficiency) to get by using conventional methods such as Direct Binary Search (DBS). Comparing with another result obtained our another system that only includes the encoder network module-1301, decoder network module-1401and the adversarial module-1500for training process, but does not include the encoder network module-2301′, decoder network module-2401′ adversarial module-2501′, the results according to the present invention show a significant improvement from our previous system.

FIG.2shows the overall structure of the neural network model (device generating module)200. The model200is constructed with six parts: two encoders (301,301′), two decoders (401,401′) and two adversarial blocks (501,501′) (as shown in theFIG.100). Encoder #1 and #2 (30) have the same structure and share the same weighs. Decoder #1 and #2 (401) have the same structure and share the same weights. Same standard applies to the adversarial blocks (501) as well. The encoder #1 (301) is configured to extract the input pattern features (801) and represent it using probability distribution which is defined as latent variable (806). The decoder #1 (401) has the similar structure to the encoder #1 (301) but in a reverse order. The decoder #1 is configured to generate the device pattern with the latent variable and the encoded condition (807). For the second decoder-encoder set: The decoder #2 (401′) takes the standard Gaussian samples along with the encoded condition (801) to generate a second output patter (811) which will be used later in the loss function. The encoder #2 (301) then takes the second output pattern (811) to generate the second latent variable set (506). Output pattern #2 and Latent Variable #2 are only used for the training (to calculate the loss function). For the final model, we will use the trained decoder (401).

FIG.3shows the detailed structure of the encoders301,301′, in which the encoders301and301′ have an identical structure. The encoder301is constructed by two convolutional layers (303&304), (one has 8 channel the second one has 16 channel), followed by two parallel Multilayer Perceptron (MLP) layer (304&305). Each of the two parallel MLP layers may be parallel fully connected layers. In some cases, each of the two parallel MLP layers includes more than two inputs. For instance, the two parallel MLP layers are configured to have 800→60 input→output dimensions, to generate the extracted pattern (mean (μ) and covariance (σ) for the Gaussian distribution). In order to get the latent variable, the re-parametrization (306) of the mean and the covariance needs to be applied. The equation of the re-parameterization is shown below in Equation 1:

where the N is the random number which obey the standard Gaussian distribution (with mean of 0 and covariance of 1).

FIG.4shows the detailed structure of the decoders401and401′, in which the decoders have an identical structure. The latent variable from the encoder (806) and the encoded condition data (807) are concatenated to form the input for the decoder (808). Then the combined data are fed into the decoder to generate the pattern. The decoder is combined with one Multilayer perceptron (MLP) layer (402) and two convolutional layers (403&404). The MLP layer has the input→output dimension of 69→800. In some cases, each of the two convolutional layers may be designed to have more than 2 channels. For instance, the two convolutional layers have the following specs: first one (403) has 8 channel the second one (404) has 16 channels. The output of second convolutional layer is the generated (or reconstructed) pattern (107). The final model used to generate different devices.

FIG.5shows the detailed structure of the adversarial blocks501and501′, in which the adversarial blocks have an identical structure. The adversarial block501has two MLP layers (502&503) (the first one has 60→100 input→output dimensions and the second one has 100→60 input→output dimension). The output of the adversarial block is the adversarial condition (504). We add the adversarial block to isolate the latent variable from the conditions in order to fit the device distribution better.

FIG.6shows the detailed training process for the conditional Variational Autoencoder (CVAE) model600according to some embodiments of the present invention. First the data (input pattern and input condition) are taken from the dataset (601), then we process them into two channel input (301) and fed into the network. The process step is shown in700. The complete training iteration include two part. Update the CVAE network (602) and update the adversarial block (606). The first loss function, Loss1 is calculated (603) after the process described above. In the meanwhile, the second loss function Loss 3 (Equation 4) is calculated after the second process: the random Gaussian sampling data (809) is later processed (1800) and then fed into the decoder encoder block (400) to get the second latent variable (506). The final loss will be the sum of the two loss functions (Loss1 and Loss3) and it will be used to update the encoder and decoder blocks (604). The second loss function (Loss 2) is calculated (607) after every three updates of the encoders and decoders. It is used to update the adversarial blocks only (608) (shown in Equation 3).

For Loss1, the first portion is the Binary cross entropy loss, the second part is the KL divergence loss and the third part is the Mean Square Error multiplied by a constant β to reach a training balance between the first adversarial block and the main model (encoder #1-decoder #1). For Loss3 (third loss function), the first term is the MSE loss between the normal gaussian sampling variable (809) and the second latent variable (506). The second term is the Mean Square Error multiplied by a constant α to reach a training balance between the second adversarial block and the main model (decoder #2-encoder #2). For Loss2, the loss function is the MSE loss between s ands, and that between s ands1. The network update in phase 1 is based on Loss1 and Loss3. By update the weight in the encoders and the decoders, we want to minimize Binary Cross Entropy Loss between the input (702) and the output pattern (107). We also want to minimize the MSE loss between the Standard Gaussian samples (809) and the latent variable #2 (812) In the meantime, the difference between the condition (701) and the two adversarial conditions (505) (506) needs to be maximized so that the encoder only extracted the pattern of the input pattern. The network update in Phase 2 is based on Loss2. In this phase, only the encoder blocks and the adversarial blocks are used and only the weight parameters in the adversarial blocks is updated (608). Here the loss is the MSE Loss between the between the condition (701) and the adversarial conditions (505), (506). By updating the adversarial blocks, we want to minimize the MES loss to form an adversarial relation between the two blocks. In order to achieve the balance between the two phases, Phase 1 updates three times while Phase 2 updates once. In order to do that, we introduce a variable n with initial value of 0. Every time the CVAE block is finished update (604), we check the n value (605). If n is smaller than 3, we add 1 to n and go back to step. If the n is 3, we feed data to update the adversarial blocks (606) and rest n to 0 (through609). In other words, the first loss function may be expressed by combination of the binary crossentropy (BCE) Loss of between the input (801) and output (107) of the first encoder decoder set and the Kullback-Leibler Divergence (KL-Divergence) between the encoded latent (806) and the standard Gaussian Distribution and the encoded latent (806) and the output of the first adversarial block (505). Further, the second loss function may be expressed by combination of Mean Square Root Loss (Mean Squared Error Loss: MSE Loss) between the encoded latent (806) and the output from the first adversarial block (505) and the MSE loss between the condition between the encoded latent (812) and the output from the second adversarial block (813), and the third loss function may be expressed by combination of the MSE Loss between the standard gaussian samples (810) and the second latent variables (812) and the MSE loss between the encoded laten (812) and the output of the adversarial block (813).

FIG.7shows the details to process the input data. The input data fed into the network (please express by a specific name of the network) (ACVAE with cycle consistency), are constructed by two channels (two 20×20 matrix) (801), the first one is the 20×20 input pattern (702) and the second is the decoded (3×20) input condition (701).

FIG.8shows the dataflow800within the network according to some embodiments of the present invention. The input pattern data (702) and the input condition data (803) forms a 2-channel input into the CVAE encoder (301). The encoder #1 then generates the latent variable #1 with dimension of 60. Then the latent variable is concatenated with the encoded condition (807) (with dimension of 9) to form the input to the decoder #1 (808). After being processed by the decoder #1, the generated (or reconstructed) pattern #1 (107) is the output. In the meanwhile, the encoded condition is concatenated with Random gaussian sampling (809) through (1700) and is fed into decoder #2 to get the output pattern #2 (811). Then output pattern #2 will be the input of encoder #2 to get the latent variable #2.

In order to fully train the Neural network model200(if number is wrong, please express by a specific name of the network) (ACVAE with cycle consistency). We used the concept of active learning.FIG.9shows the flowchart of that process. What we are doing is to train a preliminary use the original 15,000 binary training data. After finishing the first model. We use it to generate 1,000 devices with different holes sizes and label them with their spectrum at each port (condition) through the FDTD simulation. After that, we combine the new generated data along with the existing data to form a new 16,000 dataset and retrain the model. Then the second model is the final one that is used for the device generation.

FIG.10shows a mapping algorithm1,000for drawing a layout of the actual device. The CVAE generated pattern (generated hole vector107) is a 20×20 floating point matrix. Each point represents the Bernoulli distribution for each position point. So, the final generated patterns use the variable size instead of binary data to better represent the hole pattern and each floating number in the matrix will be treated as differential hole diameters. For instance, the maximum diameter of the hole may be 76.5 nm. For easier fabrication, a threshold 0.3 may be set to eliminate any holes that have the diameter below (90*0.85*√{square root over (0.3)} nm) in step (1004). Once the matrix is generated and retrieved (1001), the mapping algorithm1000is configured to create two indexes i and j to sweep through the entire matrix (starting from i=j=0). Then the algorithm1000goes to the element in the matrix Aij(1002) and check if the value is smaller than 0.3 (1003), if it is smaller, then no holes will be created. Otherwise, a hole with the diameter of: 90*0.85*√{square root over (Aij)} nm will be created on the layout of the device. We will first do the column sweeping (1005) then the row sweeping (1006) to finish the complete process. Once the sweep is done, the final layout will be the output (109).

FIGS.11a-11cshows an optical device110obtained by using the system100according to some embodiments of the present invention. The optical device110is a square based optical power splitter. The mechanism of splitting the power (of an optical beam) is to draw different holes and use the difference in the refractive index to guide the light propagation. The device110is designed for TEO mode use. The square structure has footprint of 2.5 um×2.5 um with the oxide cladding. The waveguide has a width of 500 nm and the height of 220 nm. The hole spacing is 130 nm, and the minimum and maximum hole diameter are 76.5 nm, and 42 nm, respectively.

The power splitter110is formed of nanostructured segments that are arranged in the guide material to effectively guide the input optical beam along predesigned beam paths toward the output ports. In this case, the nanostructured segments are the nanostructured hole that have a refractive index being less than that of the guide material of the power splitter. The Waveguide of the power splitter110is Silicon and the material of the nanostructured hole is Silicon Dioxide (SiO2).

FIG.12a,FIG.13a,FIG.14aandFIG.15ashow schematics illustrating the power splitters with different splitting ratios (5:5, 6:4, 7:3, and 8:2) that are generated by the fully trained model. As shown in the figures, the splitters include an input port configured to receive an input beam having an input power, a power splitter including perturbation segments arranged in a first region and a second region of a guide material having a first refractive index, each segment having a second refractive index, wherein the first region is configured to split the input beam into a first beam and a second beam, wherein and the second region is configured to separately guide the first and second beams, wherein the first refractive index is greater than the second refractive index, and output ports including first and second output ports connected the power splitter to respectively receive and transmit the first and second beams.

FIG.12b,FIG.13b,FIG.14bandFIG.15bshow the beam propagation through the devices. We consider the total power at port 1 and port 2 is 100%. The 5:5 splitter meaning port 1 holds 50% of the total output power and the port 2 holds 50% of the total output power. The 6:4 splitter meaning port 1 holds 60% of the total output power and the port 2 holds 40% of the total output power. The 7:3 splitter meaning port 1 holds 70% of the total output power and the port 2 holds 30% of the total output power. The 8:2 splitter meaning port 1 holds 80% of the total output power and the port 2 holds 20% of the total output power.

FIG.12c-12d,FIG.13c-13d,FIGS.14c-14dandFIGS.15c-15dshow the spectrum response of those devices. The devices that are generated through our Adversarial Conditional Autoencoder have very good performance (with around 90% in total transmission) across the 550 nm bandwidth (from 1250 nm to 1800 nm).

FIG.16shows a computer-implemented method1600for designing a device according to some embodiments of the present invention. The method1600includes the data process procedure of the input condition701(input data/parameters) before feeding into decoder #2. The Random Gaussian sampling variables (809) is concatenated with the encoded condition (807) to form a 3×24 matrix (1601) and it will be fed into decoder #2 (301).

According to some embodiments of the present invention, there are the following advantages with respect to the device generated from the model. The devices can be manufactured in very compact sizes. For instance, the footprint can be only 2.25 um×2.25 μm or less, which is the smallest splitter according to our knowledge. With such compact size, it has the potential to be massively integrated in optical communication chips with relatively low area budget.

The devices designed according to embodiments of the present invention can operate on an ultra-wide bandwidth (from 1250 nm to 1800 nm) while maintaining an excellent performance (around 90% transmission). Accordingly, the devices can cover all the optical communication band (From 0 band to L band corresponding to wavelengths ranging from 1260 nm to 1625 nm). SeeFIG.17. The figure indicates the properties of transmission loss of optical fibers as a function of wavelengths.

The model has been proved to generate any devices that the user wants instantly without further optimization, which significantly saves the designing time.

Note that so far nanophotonic devices with periodic holes have been described as examples. However, there are other types of optical devices. For example, the adjoint method can optimize a greater number of parameters in general. This invention can use these types of devices as training data.