RESONATORS-BASED PROGRAMMABLE OPTICAL NEURAL NETWORKS

Systems and methods are provided for devices and methods for implementing an optical neural network (ONN) by leveraging resonator structures, such on micro-ring resonators (MRRs). Examples include unit cells configured to perform a linear transformation on optical signals. Each unit cell comprises a plurality of signal mixing components optically coupled to between adjacent waveguides, where each signal mixing component corresponds to a distinct wavelength and is configured to mix optical signals on the adjacent waveguides at the distinct wavelength. Each unit cell also includes a plurality of phase tuning components each corresponding to a distinct wavelength and configured to adjust a phase of a mixed optical signal at the distinct wavelength.

BACKGROUND

Artificial neural networks (“ANNs”) have become a substantial tool for machine learning. ANNs benefit from application in a wide range of fields, such as, but not limited to, image recognition, natural language processing, and gaming, among others. For more complicated tasks, larger and deeper ANNs are becoming more important. These larger and deeper ANNs require faster and more energy-efficient computing hardware.

DETAILED DESCRIPTION

Examples of the presently disclosed technology provide for devices and methods for implementing an optical neural network (ONN). Examples disclosed herein provide optical devices, systems and methods that can be implemented for fully-connected layers in an ONN based on micro-ring resonators (MRRs). The present disclosure provides unit cells comprising one or more MRRs that can be implemented to perform linear transformations according to weights of a trained ONN. Each unit cell can be coupled between two adjacent waveguides. Each unit cell comprises signal mixing components for mixing optical signals propagating on the two adjacent waveguides and phase tuning components for adjusting a phase of the resulting mixed optical signal. The mixing and phase tuning can be adjusted to achieve a desired linear transformation.

As alluded to above, ANNs can be a tool for machine learning (ML). As ANNs become larger and deeper, they require faster and more energy-efficient computing hardware. Specifically designed computing hardware platforms, such as graphical processing units (GPUs) and tensor processing units (TPUs), have been developed to address this growing demand for computation resources. However, conventional electronic ANNs have several limitations, including high-energy consumption, limited processing speed, and susceptibility to electromagnetic interference. As a result, ONNs have drawn increasing interest because they rely on light to perform computations and processing, rather than electronic signals. The reliance on light can overcome the above bottlenecks of electronic ANNs. For example, ultra-high-speed optoelectronic devices, such as photodiodes and modulators, can enable operation rates that are one to two orders of magnitude higher than electronic counterparts. Further, the reliance on light can provide ONNs that are resistant to electromagnetic interference, have no length-dependent resistance, and consume low energy. Thus, ONNs have been recognized as a competitive solution due to ONNs offering energy-efficient, compact, and high-speed ML accelerators.

A component of ANNs are fully-connected (FC) layers. FC layers are abstraction layers of a neural network in which each neuron applies a linear transformation to an input vector through a weights matrix. As a result, all possible connections from one layer to the next layer are present, such that every input of an input vector into the one layer can influence every output of an output vector from the next layer.

In ONNs, FC layers can be realized by programmable photonic circuit architectures. Conventionally, Mach-Zehnder Interferometers (MZIs) have been used as building blocks for these programmable photonic circuits. In this case, each MZI can provide arbitrary control over power splitting ratio and relative phase shift between input and output ports of the MZI by varying relative phase between branches of the MZI. By cascading MZIs in a mesh fabric, the MZI mesh can be used to perform complex ML tasks. However, the driving power of an MZI can be relatively high. For example, dynamic power consumed to train an MZI based ONN can be 200 fJ/bit per MZI, while static power consumption, after training, can be 30 mW per MZI. Additionally, the footprint of such photonic circuits can be relatively large, generally several hundreds of μm or more in length. As a result, footprint and power consumption considerations impact fabrication of high-density ONNs that utilize MZIs. As ONNs become larger and deeper, the density increases, which results in larger device footprints and increased power consumption.

Accordingly, implementations of the present disclosure provide optical devices that can be used as building blocks of a programmable photonic circuit architecture for implementing ONNs, such as but not limited to, FC layers. As alluded to above, the present disclosure provides optical devices that utilize MRR based unit cells. These unit cells comprise one or more MRRs that are configured to perform linear transformations on optical signals at a plurality of wavelengths according to a weight matrix of a trained neural network. Relative to MZI based approaches, the unit cells disclosed herein consume less power. For example, dynamic power consumption during training may be on the order of tens of fj/bit (e.g., 10 fj/bit) per MRR and static power consumption after training may be on the order single digits of mW (e.g., 4 mW) per MRR. Thus, the unit cells according to the present disclosure can enable a wide range of applications and different matrix operations performed by the ONN.

In an example implementation, a unit cell can comprise a plurality of sub-unit cells optically coupled between adjacent waveguides. Each sub-unit cell comprises a signal mixing component and a phase tuning component, such that the unit cell comprises a plurality of signal mixing components and a plurality of phase tuning components. The plurality of signal mixing components can be configured to mix optical signals propagating on each of the adjacent waveguides for a plurality of wavelengths of light. The plurality of phase tuning components can be configured to adjust a phase of the mixed optical signal for the plurality of wavelengths. Each sub-unit is configured to operate on a distinct wavelength of light. As such, a signal mixing component and a phase tuning component of a given sub-unit are configured to operate on the corresponding distinct wavelength of light. Thus, a number of optical signals can be encoded onto the plurality of wavelengths included in a single input signal, such that each optical signal propagates at a different wavelength.

As used herein, “optically coupled” refers to an interconnection between devices to transfer an optical signal or a portion of the optical signal from one device to another. In various examples, optical coupling can be achieved by an optical coupler, such as, but not limited to, a directional couplers, through optical splitters, and the like. A directional coupler may comprise of two or more waveguides positioned close together so that light may can be evanescently transferred (e.g., by evanescent coupling) from one waveguide to another waveguide.

According to various examples of the presently disclosed technology, each phase tuning component can be provided as an MRR optically coupled to a waveguide of the adjacent waveguides. The MRR of a respective sub-unit cell can be configured to resonate at a wavelength corresponding to that sub-unit cell. A phase shifting mechanism can be coupled to the MRR and tuned so to change a refractive index of the MRR, which can be selectively controlled to vary the phase and tune the resonance of the MRR. The phase change induced by the phase shifting mechanism may be tuned so to apply a weight of a trained weight matrix to perform a linear transformation on the optical signal of a wavelength corresponding to the sub-unit. The phase shifting mechanism may be implemented as a mechanism or device that provides for thermal-optical tuning (e.g., a resistive heater coupled that generates heat based on an applied voltage), electro-optical tuning (e.g., a PN diode), metal-oxide-semiconductor capacitor (MOSCAP) tuning, or the like. Examples of a MOSCAP are provided, for example, in U.S. Pat. No. 9,612,503; U.S. patent application Ser. No. 17/972,927; and U.S. patent application Ser. No. 18/175,970, the disclosures of which are incorporated herein by reference in their entirety.

Signal mixing components, according to the present disclosure, may receive optical signals propagating on the adjacent waveguides, mix the optical signals, and output mixed, optical signal onto the adjacent waveguides, which can be used downstream. Through mixing of input optical signals for outputting onto the adjacent waveguides, each signal input into the unit cell can influence each output of the unit cell. The ratio of the mixture of the optical signals output from the unit cell can be controlled according to weights of a trained weights matrix to perform the linear transformation.

The signal mixing components may be implemented in a variety of ways. A few illustrative examples are provided herein, but other implementations are possible. In one example, a signal mixing component can be provided as a serially coupled double ring MRR having a first MRR optically coupled to a first waveguide and a second MMR optically coupled to the first MRR. The second MMR can also be optically coupled to a second waveguide that is adjacent to the first waveguide. The first and second MRRs have resonances at a common wavelength that is distinct to a respective sub-unit cell and different among the plurality signal mixing components that make up the unit cell. In another example, a signal mixing component comprises an MRR optically coupled to the first waveguide and optically coupled to the second waveguide via a contra-directional coupler. In yet another example, the second waveguide includes a plurality of waveguide crossings and a sub-unit cell comprises a MRR optically coupled to the first waveguide and to a distinct waveguide crossing of the plurality of waveguide crossings.

In each implementation of the signal mixing components, the MRRs that make up a signal mixing component may comprise phase shifting mechanisms that can be tuned to ensure that the resonance of the MRRs matches a wavelength corresponding to the sub-unit cell so to optically couple optical signal propagating on each waveguide at a corresponding wavelength into the signal mixing component, mix the optical signals, and optically couple mixed optical signals onto the waveguides. However, in some cases, the phase shifting mechanisms may dominate the energy consumption of the unit cell and, in the case of a double MRR structure, may complicate the tuning as both MRRs may need to be individually tuned to match. Thus, the implementations of signal mixing components comprising fewer MRRs may be utilized to reduce the number of phase shifting mechanism, which translates to reduced energy consumption and reduced complexity in operation.

In some examples according to the present disclosure, a ONN can be implemented as a MRR based mesh fabric comprising a plurality of cascaded unit cells, as described above. As a result of the MRR based unit cells according to the present disclosure, a mesh fabric can be achieved having a footprint that is at least 10 times smaller than the conventional MZI based approach, where a MZI based unit cell may be 100 μm by 400 μm. In the MRR based mesh fabric, a plurality of waveguides can be provided and building blocks or unit cell comprising a plurality of signal mixing components and a plurality of phase tuning component are optically coupled between adjacent waveguides. The MRR-based unit cells can incorporate wavelength-division multiplexing (WDM) techniques, which enable multiplication with different matrices, through sub-unit cells configured for distinct wavelengths of light. This approach can be leverage to decrease computation time through increased computation speeds of matrix multiplication functions of an ONN, while also providing for ONNs of increased density with reduced footprints. Moreover, by using WDM and exploiting multiple free spectral ranges (multi-FSRs) of the MRRs, matrix-vector multiplications (MVMs) of a single photonic core can be extended into multiple general matrix multiplication (GEMM) operations by parallel processing within a single optical device.

FIG.1depicts a schematic block diagram of a unit cell100in accordance with the presently disclosed technology. Unit cell100can receive a first input signal140avia a first input port104aand a second input signal140bvia a second input port104b. Each of the first and second input signals140aand140bmay comprise a plurality of optical signals propagating at a plurality of wavelengths of light. In various examples, first input signal140aand second input signal140bcomprise the same plurality of wavelengths of light. Each optical signal of each wavelength for the first input signal140aand second input signal140bmay be encoded with information. For example, an optical signal of a first wavelength of first input signal140amay be encoded with first information (e.g., data of a first element of a vector or matrix), an optical signal at a second wavelength of light of first input signal140amay be encoded with second information (e.g., data of a second element of a vector or matrix), and so on. Similarly, an optical signal at a first wavelength of light of second input signal140bmay be encoded with the first information (e.g., data of a first element of a vector or matrix), an optical signal at a second wavelength of light of second input signal140bmay be encoded with the second information (e.g., data of a second element of a vector or matrix), and so on. The unit cell100can be configured to apply a weight to the information encoded onto first and second input signals140aand140bby mixing the input signals and tuning a phase of the mixed signal according to a trained weight programmed into the unit cell100. The weighted information, as a result of the mixing and phase tuning, can be encoded on the resulting output signals, which is output from first output port106aas first output signal150aand second output port106bas second output signal150b.

Unit cell100comprises a plurality of sub-unit cells110a-110n(individually referred to herein as a sub-unit cell110as an illustrative example or collectively referred to herein as sub-unit cells110). The sub-unit cells110comprise signal mixing components120a-120n(individually referred to herein a signal mixing component120as an illustrative example or collectively referred to herein as signal mixing components120) and phase tuning components130a-130n(individually referred to herein as a phase tuning component130as an illustrative example or collectively referred to herein as phase tuning components130). For example sub-unit cell110aincludes a signal mixing component120aand a phase tuning component130a, sub-unit cell110bincludes a signal mixing component120band a phase tuning component130b, and so on.

The signal mixing components120can be configured to mix first and second input signals140aand140b. Each sub-unit cells110can be associated with a distinct wavelength of light of the plurality of wavelengths. The signal mixing components120of a respective sub-unit cells110can be configured to mix an optical signal of first input signal140aof the associated wavelength with an optical signal of second input signal140bof the same wavelength. For example, sub-unit cell110amay be associated with a first wavelength of light, and sub-unit cell110bmay be associated with a second wavelength of light. First and second input signals140aand140beach comprise information encoded on the first wavelength and information encoded onto the second wavelength. Signal mixing component120acan be configured to mix optical signals from first and second input signals140aand140bthat propagate at the first wavelength, and signal mixing component120bcan be configured to mix optical signals from first and second input signals140aand140bthat propagate at the second wavelength.

The phase tuning components130can be configured to adjust a phase of the mixed input signals140aand140bfor each respective wavelength of the plurality of wavelengths. Each phase tuning component130can be configured to operate on a wavelength of light associated with a respective one of the sub-unit cells110. With reference to the above example, phase tuning component130acan be configured to adjust a phase of a mixed signal, which is output from the signal mixing component120a, at the first wavelength. Phase tuning component130bcan be configured to adjust a phase of a mixed signal, which is output from the signal mixing component120b, at the second wavelength.

As a result, a number of input optical signals can be encoded onto the plurality of wavelengths included in the input signals140aand140b, such that each optical signal propagates at a different wavelength corresponding to a given signal mixing component120and a given phase tuning component130. The phase tuning components130can be used to control the relative phase differences among different ports (e.g., second input port104band first input port104a) independently, the signal mixing components120can be implemented to connect and mix signals of the different ports.

FIG.2depicts an example implementation of a unit cell in accordance with the presently disclosed technology.FIG.2depicts unit cell200, which is an example implementation of unit cell100. Unit cell200comprises a plurality of sub-unit cells, illustratively shown as sub-unit cell210athrough sub-unit cell210c(individually referred to herein as a sub-unit cell210as an illustrative example or collectively referred to herein as sub-unit cells210). Sub-unit cell210athrough sub-unit cell210ccomprise signal mixing component220athrough signal mixing component220c(individually referred to herein as a signal mixing component220as an illustrative example or collectively referred to herein as signal mixing components220) and phase tuning component230athrough phase tuning component230c(individually referred to herein as a phase tuning component230as an illustrative example or collectively referred to herein as phase tuning components230). Signal mixing components220and phase tuning components230are example implementations of signal mixing components120and phase tuning components130, respectively. While three sub-unit cells210are shown inFIG.2, this is for illustrative purposes only. Any number of sub-unit cells210may be included in unit cell200.

Unit cell200is optically coupled between adjacent waveguides, shown as first waveguide202aand second waveguide202b(individually referred to herein as a waveguide202or bus waveguide202as an illustrative example or collectively referred to herein as waveguides202or bus waveguides202). Waveguides202aand202bmay be formed of semiconductor material, such as silicon or other Group IV material or Group III-V material. First and second input signals240aand240bcan be injected into the first and second waveguides202aand202b, respectively. The first and second input signals240aand240bmay be substantially similar to first and second input signals140aand140b. As such, first and second input signals240aand240bcomprise a plurality of optical signals propagating at a plurality of wavelengths, where each optical signal is encoded with information. First and second output signals250aand250bare output from first and second waveguides202aand202b, respectively.

Sub-unit cells210can be implemented as resonator structures. For example, signal mixing component220acomprises a first resonator structure222aoptically coupled to first waveguide202aand a second resonator structure224aoptically coupled to second waveguide202b. The first resonator structure222ais optically coupled to the second resonator structure224a. The configuration of first resonator structure222aand second resonator structure224acan be referred to as a serially coupled double resonator structure. In the case of signal mixing component220b(or signal mixing component220c), first resonator structures222b(or first resonator structure222c) and second resonator structures224b(or second resonator structure224c) are optically coupled together and optically coupled to the first waveguide202aand second waveguide202b, respectively. First resonator structures222a-222care individually referred to herein as a resonant structure222as an illustrative example or collectively referred to herein as first resonator structures222. Second resonator structures224a-224care individually referred to herein as a resonate structure224as an illustrative example or collectively referred to herein as second resonator structures224. In the example ofFIG.2, the resonator structures222and224are provided as MRRs forming serially coupled double ring MRR structures.

Like the signal mixing components220, the phase tuning components230comprise at least one resonator structure optically coupled to second waveguide202b. For example, phase tuning components230a-230ccomprise a resonator structure232a, resonator structure232b, and resonator structure232c(individually referred to herein as a resonate structure232as an illustrative example or collectively referred to herein as resonator structures232), respectively, optically coupled to the second waveguide202b. In various examples, a respective resonator structure232of a given phase tuning component230is optically coupled to second waveguide202bdownstream of a corresponding signal mixing components220. In the case ofFIG.2, the phase tuning component230ais positioned between the comprising signal mixing component220aand a signal mixing component220b. However, a phase tuning component230amay be positioned downstream of the signal mixing component220b, according to some implementations.

As described above, the resonator structures222,224, and232may be implemented as MRRs, in some examples, or other optical ring resonator. A resonator structure may be a closed loop waveguide formed of semiconductor material, such as silicon or other Group IV material. The shape of the loop may be, for example, but not limited to, circular, elliptical, a obound shape, etc., thereby forming a resonator structure. A resonator structure may have an initial resonance wavelength (A) defined by the circumference length of the resonator structure (e.g., a radius in the case of an MRR). A resonator structure also comprises a plurality of resonance frequencies separated by an integer number of FSRs of the resonator structure (e.g., λ+(k−1)Δλ, where k is a non-zero integer).

In various examples, the resonator structures222,224, and232evanescently couple to another waveguide (e.g., a bus waveguide or another resonator structure) by use of a directional coupler. In the case of evanescent coupling, the amount of light optically coupled into one waveguide to another is based on a distance between the waveguides and a resonance frequency of the resonator structure. The amount of light optically coupled can be represented by a coupling coefficient or coupling ratio. The distance between waveguides can be controlled so to bring one waveguide as close as possible to another waveguide. The coupling length represents an effective curve length of the resonator structure for the coupling phenomenon to happen with the waveguide, which can be based on a radius of curvature of the resonator structure where the evanescent coupling is to occur. In the case ofFIG.2, the radius of the MRR structures may be designed to achieve a desired curve length for a desired resonance frequency. The resonance frequency is also dependent on effective refractive indices between the waveguide and the optical ring resonator.

An input optical signal can be optically coupled into and out of a resonator structure based on the resonance frequency of the resonator structure. For example, an input optical signal propagating at a wavelength aligned with a resonance frequency of the resonator structure can be optically coupled into the resonator structure, while other wavelengths are agnostic to the resonator structure. Thus, the resonance frequency of resonator structures included in the sub-unit cells210may be tuned to configure each of the sub-unit cell210to operate on a distinct wavelength of light, thereby associating a distinct wavelength of light to each sub-unit cell210.

For example, in the case ofFIG.2, first input signal240aand second input signal240bmay include optical signals propagating at a first wavelength, a second wavelength, and a third wavelength. The resonance frequency of resonator structures222a,224a, and232aof sub-unit cell210amay be aligned with the first wavelength. Thus, optical signals at the first wavelength can be optically coupled into and output from the first resonator structure222aand second resonator structure224a, as well as the resonator structure232a. Similarly, the resonance frequency of resonator structures222b,224b, and232bmay be aligned with the second wavelength, such that optical signals at the second wavelength can be optically coupled into and output of the resonator structures222b,224b, and232b. Likewise, the resonance frequency of resonator structures222c,224c, and232cmay be aligned with the third wavelength, such that optical signals at the third wavelength can be optically coupled into and output of the resonator structures222c,224c, and232c.

For a given phase tuning component230, the electric field transmission function (TPT_Double_MRR) of an optical signal input into the resonator structure232from second waveguide202band an optical signal output onto the waveguide second waveguide202bafter passing through the resonator structure232is provided below.

where a is the single round trip amplitude transmission in the resonator structure, which is determined based on the material of the waveguide forming the resonator structure and bending losses in the resonator structure; t is the transmission coefficient, which is based on to the coupling coefficient (kpt) between resonator structure232and second waveguide202bas t2+kpt2=1; θ is the phase of the light propagating in the resonator structure; and j is an imaginary number. The phase (θ) of light in the resonator structure can be provided as θ=2πfn(L/c), where f represents the resonance frequency, n is refractive index of the resonator structure; L is the circumference length of the resonator structure; and c is the speed of light.

The resonator structure232can be tuned on/off resonance by varying the phase θ, which can be achieved by changing the refractive index of the waveguide forming the resonator structure232. Resonator structures232a-232ccan include phase shift mechanisms234a-234c(individually referred to herein as a phase shift mechanism324as an illustrative example or collectively referred to herein as phase shift mechanisms234) disposed thereon, respectively configured to alter the effective index of refraction of a waveguide of the respective resonator structure232. The phase shifting mechanisms234can be implemented through thermal-optical tuning (e.g., a resistor coupled to the waveguide that generates heat based on an applied voltage), electro-optical tuning (e.g., coupling a PN diode to the waveguide), metal-oxide-semiconductor capacitor (MOSCAP) tuning, or the like. Phase shifting mechanisms234can be independently or collectively controlled, as described below, to alter the effective refractive index of a respective resonator structure, thereby tuning the resonance of the respective resonator structure.

The phase tuning components230may operate in an over-coupled regime. For example, the resonator structure232may be over-coupled to second waveguide202b. As used herein “over-coupled” refers to evanescent coupling between waveguides having a relatively high coupling coefficient (k). The coupling coefficient (k) is a value between 0 and 1, where a value of 1 expresses perfect or 100% coupling of light from one waveguide to another waveguide. A high coupling coefficient may be, for example, a power coupling coefficient (k2) that is greater than the single round trip loss (1−a2) in the resonator structure232(e.g., k2>1−a2or tea). In an example implementation, k2is set as 0.2 (e.g., k is approximately 0.45) and a is 0.953 (e.g., a2=0.907). Any coupling coefficient is possible as long as it satisfies the above conditions. As discussed above, the coupling coefficient is dependent upon the distance between waveguides, as well as the resonance frequency of the resonator structure232.

FIGS.3A and3Bdepict an example transmission response and phase response of a phase tuning component in accordance with an example implementation disclosed herein.FIG.3Ashows a transmission response plot310andFIG.3Bshows a phase response plot320at the output of an example phase tuning components230, where a normalized transmission and a phase shift in radians is plotted as a function of phase detuning in radians. The phase detuning refers to a detuning of the resonance frequency of a phase tuning component230by the phase shifting mechanism, while the phase shift refers to a change in the phase of the light in the resonator structure after tuning relative to the phase without tuning. Transmission response plot310depicts a transmission response312of an ideal all-pass filter, where the transmission response remains at unity. Transmission response plot310also depicts a transmission response314of an example implementation of a phase tuning components230, where the coupling coefficient (k) of resonator structure232to the second waveguide202bis approximately 0.45. Phase response plot320depicts a phase response324of the example implementation of phase tuning components230. As can be seen fromFIGS.3A and3B, a phase shift can be achieved by detuning phase tuning components230, with minimal effect on the transmission (e.g., amplitude) of the output signal. In this example, the transmission loss is at most 5%.

While the phase tuning components230can be used to control the relative phase differences among different waveguides independently, the signal mixing components220can be implemented to connect and mix signals from different waveguides. Referring back toFIG.2, as described above, signal mixing components220can be implemented as a serially coupled double resonator structure. This configuration can be used to couple signals in different waveguides that propagate along a common direction and at a common wavelength, as well as enabling the cascading of multiple components with mesh setups for multiple-port linear transformation, as described below in connection withFIGS.8A,9A, and10A, while keeping the bus waveguide straight without a need for waveguide bend or crossing.

Referring back toFIG.2, in an example implementation, resonator structures222and resonator structures224of a given signal mixing component220may be identical. For example, resonator structures222and resonator structures224may have the same circumference length, waveguide material, and resonance frequency. Resonator structures222and resonator structures224may have the same separation distance between its waveguide and first waveguide202aand second waveguide202b, respectively. As such, resonator structures222and resonator structures224may have the same coupling coefficient (k0). The coupling coefficient between resonator structures222and resonator structures224(denoted as k1) may be configured to provide a flat optical response from input to output, which means that resonator structures222couples to resonator structures224with minimal or no loss between input and output. If the optical response is not flat, a loss in the response may occur which can be difficult to tune. Thus, implementations disclosed herein utilize a coupling coefficient (k1) that is as high as possible while maintaining a flat optical response. In an example implementation, k12is set as 0.013 (e.g., k1is approximately 0.114), which provides the flat optical response. The electric field transmission matrix (TMT_Double_MRR) (also referred to as transfer matrix) of the example signal mixing components220ofFIG.2is:

where a is the single round trip amplitude transmission in the resonator structure, which is the same for resonator structures222and resonator structures224; t is the transmission coefficient, such that t2represents a transmission coefficient between first waveguide202aand resonator structures222, t1represents a transmission coefficient resonator structures222and resonator structures224, and to represents a transmission coefficient between second waveguide202band resonator structure224; e is the phase of the light propagating in the resonator structures, which is controlled so to be the same for resonator structures222and resonator structures224. In Equation 2, k2represents the coupling coefficient between first waveguide202aand resonator structures222and k1represents the coupling coefficient between first waveguide202band resonator structure224. As noted above, in some examples, the resonator structures224and222maybe substantially identical, in which case k2may be the same or approximately the same as k0. However, implementations disclosed herein are not limited to identical resonator structures, and as such k0and k2may differ for different applications. For similar reasons, to and t2may be the same or different.

To control the phase θ, the refractive index of the waveguide forming the resonator structures222and resonator structures224can be altered by utilizing phase shift mechanisms, similar to phase shift mechanisms234discussed above. For example, first resonator structure222aand second resonator structure224amay comprise first phase shift mechanism226aand second phase shift mechanism228a, respectively, disposed thereon and resonator structures222band resonator structures224bmay comprise first phase shift mechanism226band second phase shift mechanism228b, respectively, disposed thereon. Likewise, resonator structures222cand resonator structures224cmay comprise first phase shift mechanism226cand second phase shift mechanism228c, respectively, disposed thereon. First phase shift mechanisms226a-226care collectively referred to herein as first phase shift mechanism226and second phase shift mechanisms228a-228care re collectively referred to herein as second phase shift mechanism228. The phase shifting mechanisms226and228can be configured to alter an effective index of refraction of a waveguide of the respective resonator structure. The phase shifting mechanisms226and228can be implemented through thermal-optical tuning (e.g., a resistor coupled to the waveguide that generates heat based on an applied voltage), electro-optical tuning (e.g., coupling a PN diode to the waveguide), metal-oxide-semiconductor capacitor (MOSCAP) tuning, or the like. Phase shifting mechanisms226and228can be independently or collectively controlled, as described below, to alter the effective refractive index of a respective resonator structure, thereby tuning the resonance of the respective resonator structure. By tuning the resonance, the intensity (e.g., amplitude) of light coupled into and out of each resonator structure can be adjusted. In various examples, the resonance of each resonator structures222and resonator structures224can be tuned simultaneously to have the same transmission response to the detuning.

FIG.4depicts an example transmission response of a signal mixing component in accordance with an example implementation disclosed herein.FIG.4shows a transmission response plot410at the output of an example signal mixing component220, where a normalized transmission is plotted as a function of phase detuning in radians. In this example, the coupling coefficient (k1) between resonator structure222and resonator structure224is approximately 0.114. As can be seen fromFIG.4, signal mixing component220can be utilized to achieve any intensity transmission coefficients, from 0 to 1, through phase detuning (e.g., tuning of the resonance frequencies of resonator structure222and resonator structure224), which can be utilized to implement unitary matrix using the unit cell200.

WhileFIG.2depicts resonator structure232as smaller than224and/or resonator structures222, this is for illustrative purposes only. The figures are not intended to be drawn to scale and the relative sizes between structures depicted inFIG.2(as well as other figures of the present disclosure) are not intended to be limiting. Thus, the circumference length of resonator structure232may be the same, larger, or smaller than resonator structures224and/or resonator structures222, as desired for a given application. In an illustrative example, the circumference length of resonator structure232may be approximately the same as resonator structures224and/or resonator structures222so to resonant at the same frequency as resonator structures224and/or resonator structures222. This permits resonator structure232to have the same resonance frequency as resonator structures224and/or resonator structures222, thereby operating on the same wavelength of light.

As alluded to above, phase shifting mechanisms may dominate the energy consumption of the unit cell200. For example, when a phase shifting mechanism is implemented as a resistive heater for thermal-optical tuning, the energy consumption by the phase shifters represents the majority of the energy consumed by the unit cell200, and thus dominates the energy consumption of a mesh network that implements unit cell200. Accordingly, some implementation disclosed herein can be utilized to reduce the number of phase shifting mechanisms included in a unit cell, thereby reducing energy consumption of the unit cell. Example implementations are provided below in connection withFIGS.5and6.

FIG.5depicts an example of another implementation of a unit cell in accordance with the presently disclosed technology.FIG.5depicts unit cell500, which is substantially similar to unit cell200except as provided herein. Particularly, unit cell500provides a single resonator structure implementation of signal mixing components. Reference numbers inFIG.5refer to elements of like reference numbers fromFIG.2and the description provided in connection withFIG.2applies here to like reference numbers inFIG.5.

For example, unit cell500is optically coupled between first waveguide202aand second waveguide202b. First and second input signals240aand240bcan be injected into the first and second waveguides202aand202b, respectively, and the first and second output signals250aand250bare output form first and second waveguides202aand202b, respectively.

Unit cell500comprises a plurality of sub-unit cells, illustratively shown as sub-unit cell510athrough sub-unit cell510c(individually referred to herein as a sub-unit cell510as an illustrative example or collectively referred to herein as sub-unit cells510). The sub-unit cells510each comprise a signal mixing component and a phase tuning component. For example, sub-unit cells510a-510ccomprise signal mixing component520athrough signal mixing component520c(individually referred to herein as a signal mixing component520as an illustrative example or collectively referred to herein as signal mixing components520) and phase tuning component230athrough phase tuning component230c. While three sub-unit cells510are shown inFIG.5, this is for illustrative purposes only. Any number of sub-unit cells510may be included in unit cell500. As described above, the sub-unit cells510comprise resonator structures that can be tuned so to configure each sub-unit cell510to operate on a distinct wavelength of light, thereby associating a distinct wavelength of light to each sub-unit cell510.

As noted above, each sub-unit cell510comprises phase tuning components230and the description of phase tuning components230provided in connection withFIG.2applies toFIG.5. Thus, each of the phase tuning components230comprises a resonator structure232and a phase shift mechanism234disposed thereon. A respective phase tuning component230of a given sub-unit cell510can control the phase difference between optical signals, propagating at a wavelength corresponding to the given sub-unit cell510, on first waveguide202aand second waveguide202b.

Each sub-unit cell510also comprises a signal mixing component520, which may be substantively similar to the signal mixing components220, except as provided herein. For example, signal mixing components520a-520ccomprise resonator structures224a-224c, respectively, optically coupled to second waveguide202b. In the example ofFIG.5, signal mixing components520a-520care also optically coupled to a reverse propagation coupling sections524a-524c(individually referred to herein as a reverse propagation coupling sections524or coupling section524as an illustrative example or collectively referred to herein as reverse propagation coupling sections524or coupling sections524), via waveguide crossings522a-522c(individually referred to herein as a waveguide crossing522as an illustrative example or collectively referred to herein as waveguide crossings522) of the waveguide202a. A waveguide crossing refers to a continuous waveguide that includes a section of the waveguide that crosses over another section of the waveguide. The waveguide cross exhibit low loss, such that an optical signal in one section of the waveguide does not pass (or a negligible amount passes) into the other section of the waveguide.

For example, each waveguide crossing522is formed from first waveguide202aand coupling section524. Light on first waveguide202aentering a waveguide crossing522propagates in a first direction and into coupling section524, toward the output end of the unit cell500. The first waveguide202ais turned in a reverse direction (e.g., toward the input end of unit cell500) at the coupling section524. The coupling section524is brought as close as possible to a resonator structure224so to optically couple light from the section of first waveguide202ainto the resonator structure224, and vice versa. Due to the reversed direction of the first waveguide202aalong the coupling section524, light in the coupling section524propagates in a reverse direction relative to the first input signal240a, such that the light is propagating toward the input end of unit cell500. In this way, light in waveguide crossing522can be coupled into resonator structure224, which mixes the optical signal from the second waveguide202bwith light from first waveguide202a. The mixed optical signals can then be supplied back to second waveguide202band first waveguide202a, which propagates downstream to the output end. In examples disclosed herein, propagation losses in the waveguide crossings522can be lower than those in a resonator structure, such as resonator structures224, for example, less than 0.1 dB per crossing.

The electric field transmission matrix (TMT_Signle_MRR) of the example signal mixing components520ofFIG.5is:

where a is the single round trip amplitude transmission in the resonator structure224; t is the transmission coefficient, such that to represents a transmission coefficient between second waveguide202band resonator structure224; θ is the phase of the light propagating in the resonator structure224; j is an imaginary number; and k0is the coupling coefficient between second waveguide202band resonator structure224. In this example, the coupling k0also represents the coupling coefficient between resonator structure224and first waveguide202aat the coupling section524. However, in other implementations, the coupling between resonator structure224and second waveguide202bneed not be the same as the coupling between resonator structure224and coupling section524.

As described above in connection withFIG.2, the refractive index of the waveguide forming the resonator structures224can be altered through control of the phase θ by utilizing phase shift mechanisms228. By tuning the resonance, the intensity of light coupled into and out of resonator structure224can be adjusted.

FIG.6depicts an example of another implementation of a unit cell in accordance with the presently disclosed technology.FIG.6provides unit cell600, which is substantially similar to unit cell200except as provided herein. Particularly, similar to unit cell500above, unit cell600provides a single resonator structure implementation of signal mixing components. Reference numbers inFIG.6refer to elements of like reference numbers fromFIG.6and the description provided in connection withFIG.2applies here to like reference numbers inFIG.6.

For example, unit cell600is optically coupled between first waveguide202aand second waveguide602b. Second waveguide602bmay be substantively similar to second waveguide202b, except that second waveguide602bmay include taper604and inverse taper606. Taper604increases the width of the second waveguide602bfrom a first width to a second width (wd), while the inverse taper606decreases the width from the second width to a third width, which may be the same or different from the first width. In various examples, the first and third widths are smaller than the second width (wd). First and second input signals240aand240bcan be injected into the first and second waveguides202aand202b, respectively, and first and second output signals250aand250bare output form first and second waveguides202aand202b, respectively.

Unit cell600comprises a plurality of sub-unit cells, illustratively shown as sub-unit cell610aand sub-unit cell610b(individually referred to herein as a sub-unit cell610as an illustrative example or collectively referred to herein as sub-unit cells610). The sub-unit cells510each comprise a signal mixing component and a phase tuning component. For example, sub-unit cells610aand610bcomprise signal mixing component620aand signal mixing component620b(individually referred to herein as a signal mixing component620as an illustrative example or collectively referred to herein as signal mixing components620) and phase tuning components230aand230b. While two sub-unit cells610are shown inFIG.6, this is for illustrative purposes only. Any number of sub-unit cells610may be included in unit cell600. As described above, the sub-unit cells610comprise resonator structures that can be tuned so to configure each sub-unit cell610to operate on a distinct wavelength of light, thereby associating a distinct wavelength of light to each sub-unit cell610.

As noted above, each sub-unit cell610comprises phase tuning components230and the description of phase tuning components230provided in connection withFIG.2applies toFIG.6as well. Thus, the phase tuning components230each comprise a resonator structure232and a phase shift mechanisms234disposed thereon. Each phase tuning component230of a given sub-unit cell610can control the phase difference between optical signals, propagating at a wavelength corresponding to the given sub-unit cell610, on first waveguide202aand second waveguide602b.

Each sub-unit cell610also comprises a signal mixing component620, which may be substantively similar to the signal mixing components220, except as provided herein. For example, signal mixing components620aand620bcomprise first resonator structures224aand224b, respectively, optically coupled to first waveguide202aand to the second waveguide602b.FIG.6illustrates directional coupler625aand625b(individually referred to herein as a directional coupler625as an illustrative example or collectively referred to herein as directional couplers625) implemented to provide for coupling light between a respective resonator structures224and second waveguide602b. Each directional coupler625can provide for evanescent coupling of light from second waveguide602binto a respective resonator structures224and vice versa based on the coupling coefficient (k0) therebetween as well as the resonance frequency of the resonator structures224. The third width (wd) of second waveguide602bwithin the directional coupler625a(and similarly within directional coupler625b) may be approximately the same as a width (wc) of the waveguide forming the second resonator structure224a(and similarly resonator structures224b). This configuration may assist in coupling due to phase mismatch between optical signals on the second waveguide602brelative to an optical signal on resonator structure224.

In the example ofFIG.6, second resonator structure224aand resonator structures224bcouples to the first waveguide202athrough a contra-directional coupler623aand a contra-directional coupler623b(individually referred to herein as a contra-directional coupler623as an illustrative example or collectively referred to herein as contra-directional couplers623), respectively. A contra-directional coupler comprises periodic perturbations over the length L of the coupler. The periodic perturbations are configured to reflect optical signals propagating at one wavelength while transmitting optical signals of other wavelengths. For example, contra-directional coupler623amay function to reflect an optical signal propagating at the first wavelength corresponding to sub-unit cell610aand optical signals of other wavelengths to be transmitted downstream. The reflected optical signal is then coupled into resonator structure224aand propagates in the same direction (e.g., counter clockwise direction) as the optical signal coupled into second resonator structure224afrom second waveguide602b.

In an example implementation, each contra-directional coupler623may comprise a wavelength-specific reflector621that comprises a pair of approximately equal-period Bragg gratings respectively coupled to first waveguide202aand resonator structures224. The pair of Bragg gratings comprise small-amplitude perturbations (e.g., on the order of tens of micrometers) relative to the widths of the respective waveguide. As shown inFIG.6, waveguide202amay have a width waat least within the contra-directional coupler623aand the waveguide of resonator structure224amay have a width wbat least within the contra-directional coupler623a, which may be larger than wa. A first Bragg grating627may be provided on the waveguide202ahaving a grating period A and grating amplitude of h1(e.g., amplitude perturbations). A second Bragg grating629can be provided coupled to the waveguide of second resonator structure224ahaving the grating period A (which may be approximately equal to that of grating627) and grating amplitude of h2(which may be smaller than h1). The Bragg grating629on the resonator structures224amay function as an anti-reflection grating to suppress Bragg reflections of the grating627. The grating629may be provided out-of-phase relative to the grating627(e.g., perturbations of grating629aligned with perturbations of grating627). In an example implementation, wamay be 450 nm and wbmay be 550 nm, A may be 318 nm, h1may be approximately 30 nm, and h2may be approximately 40 nm. However, h2need not be larger than h1and may be smaller depending on the application. The length L of each Bragg grating627and629may be 159 nm and comprise 40 gratings. The period A of each grating determines which wavelengths are reflected, thus controlled selection of the period can be used to tune the wavelength that will be reflected by each Bragg grating pair. While the above example is provided with reference to contra-directional coupler623a, the other contra-directional coupler623may have a substantially similar configuration, except that the period A is selected according to a desired wavelength for each sub-unit cell610.

The electric field transmission matrix (TMT_Signle_MRR) of the example signal mixing components620ofFIG.6is represented by Eq. 3 above. Thus, as described above in connection withFIG.2, the refractive index of the waveguide forming the resonator structures224can be altered through control of the phase θ by utilizing phase shift mechanisms228. By tuning the resonance, the intensity of light coupled into and out of resonator structure224can be adjusted.

FIG.7depicts another example transmission response of a signal mixing component in accordance with an example implementation disclosed herein.FIG.7shows a transmission response plot710at the output of an example signal mixing component implemented using a single resonator structure, where a normalized transmission is plotted as a function of phase detuning (0) in radians. The signal mixing component may be implemented, for example, as signal mixing components520or signal mixing components620. As can be seen fromFIG.7, signal mixing component can be utilized to achieve any intensity transmission coefficients, from 0 to 1, through phase detuning (e.g., tuning of the resonance frequencies of resonator structures224, which can be utilized to implement unitary matrix using the unit cell500and/or600.

FIGS.8A-10Bare schematic diagrams of various optical systems that can be utilized to perform various multiplication processes on multiple different matrices in accordance with implementations disclosed herein.FIGS.8A-10Beach comprise an input layer configured to supply an input signal comprising optical signals encoded according to an input matrix; a weight matrix core configured to apply a weight matrix to the input signal, and an output layer configured to detect output signals comprising output optical signal encoded with an output matrix representative of the weight matrix applied to the input matrix.

A feed-forward neural network cascades multiple layers to increase the approximation capability, where information moves in one direction from an input layer to an output layer. For an ithlayer, an input and output relationship can be represented as:

where Y represents an ithoutput matrix; W represents an ithweight matrix; X represents an ithinput matrix; and f represents an element-wise nonlinear activation function. In the case of optical devices shown inFIGS.8-10, elements of Y and X may be encoded into amplitudes of optical signals included in the input signals, such as first and second input signals140a, first and second input signals140b, first and second input signals240a, and first and second input signals240b. Elements of W may be learned weights to perform a linear transformation on the input matrix X.

The different configurations shown inFIGS.8-10may be utilized to perform various matrix multiplication functions for applying a weight matrix W to an input matrix X. For example,FIG.8Adepicts an optical system800for performing multiple matrix-vector multiplication (MVMs) operation850, shown inFIG.8B.FIG.9Adepicts an optical system900for performing multiple MVMs operation950on a set of inputs, as shown inFIG.9B.FIG.10Adepicts an optical system1000for performing one or more general matrix multiplication (GEMM) operations1050, as shown inFIG.10B. In each case, the weight matrix core can be implemented by cascading a plurality of unit-cells, such as, unit cells100ofFIG.1, unit cell200ofFIG.2, unit cell500ofFIG.5, and/or unit cell600ofFIG.6.

Referring toFIGS.8A and8B, MVM operation850includes multiplying input vector852to a plurality of weight matrices854. Input vector852may be a n×1 vector and weight matrices854may be a m set of n×n weight matrices854a-854n, comprises weight values as entries, to be applied to input vector852, where m and n are integers greater than zero. Applying weight matrices854to input vector852, results an output vectors856that comprises an m set of output vectors856a-nthat are products of each matrix854a-854nand input vector852. While three weight matrices are illustrated herein (e.g., m=3), any number of weight matrices may be included in weight matrices854.

To achieve operation850, optical system800comprises input layer830that receives input signals from an optical source (such as a comb laser or other light source as known in the art). The optical source may emit a plurality of input signals832a-832nonto a plurality of waveguides822a-822n. Each input signal832a-832nmay comprises an m number of optical signals, each propagating at a distinct wavelength. The number of waveguides822a-822nmay correspond to the number of entries in the input vector852and the number of optical signals at distinct wavelengths may correspond to the number of weight matrices854a-854n.

Optical system800comprises a plurality of wideband modulators834a-834ndisposed on the plurality of waveguides822a-822n. The wideband modulators834a-834nmay be configured to modulate the amplitude of input signals832a-832naccording to entries of the input vector852. For example, wideband modulator834amay be configured to modulate amplitudes of the optical signals that make up input signal832aaccording to a first entry (x1) of input vector852. As wideband modulators, wideband modulator834acan modulate optical signals at each of the distinct wavelengths according to the first entry. Similarly, wideband modulators834b-nmodulate amplitudes of the optical signals included in input signal832b-832naccording to a second through nthentry of input vector852(e.g., x2through x4in this example). Modulating the amplitudes of the optical signals encodes each optical signal with an entry of the input vector852. The wideband modulators834a-834nmay be implemented as, for example but not limited to, a Mach-Zehnder Modulator, Electro-absorption modulator, or the like. A control circuit860may be communicatively coupled (e.g., by a wired or wireless interface) to control the modulators834aaccording to the entries of the input vector852. Control circuit860may be implemented as computer system1200, discussed below in connection withFIG.12.

The encoded input signals are supplied to a weight matrix core820(also referred to as a cross bar array). Weight matrix core820comprises a plurality of unit cells824a-824narranged in a cascaded structure. The plurality of unit cells824a-824nmay be implemented as any of unit cell100, unit cell200, unit cell500, or unit cell600. Each unit cell824a-824ncan be tuned according to entries in the weight matrices854. For example, each unit cell824a-824nmay comprise a m number of sub-unit cells, as described above in connection withFIGS.1,2,5, and6. Each sub-unit cell, which comprises one or more resonant structures as described above, may be tuned for a distinct wavelength of the m optical signals. Signal mixing components and phase tuning components of sub-unit cell can then be detuned so to apply weights according to entries of the weight matrices854.

For example, a first set of sub-unit cells of the unit cells824a-824nmay be tuned to a first wavelength, corresponding to weight matrix854a. The signal mixing components and phase tuning components of the first set of sub-unit cells can be detuned according to entries of weight matrix854a. Thus, the first set of sub-unit cells can apply the weight matrix854ato the input vector852by modulating the optical signal at the first wavelength encoded with entries of the input vector852based on the detuning of the signal mixing components and the phase tuning components. Similarly, a second set of sub-unit cells of the unit cells824a-824nmay be tuned to a second wavelength, corresponding to weight matrix854b, and an mthset of sub-unit cells of the unit cells824a-824nmay be tuned to an mthwavelength, corresponding to weight matrix854n. Thus, each unit cell comprises one sub-unit cell of each of the first, second, and mthset of sub-unit cells.

In various examples, control circuit860may be utilized to tune the unit cell. For example, control circuit860may adjust a voltage bias applied to phase shifting mechanisms coupled the resonator structures of the unit cells824a-824n. The phase shifting mechanism may be phase shifting mechanisms226, phase shifting mechanisms228, and/or resonator structure232, as discussed above in connection withFIGS.2,5, and6. The control circuit860may issue control commands to a voltage source862that applies a voltage bias to the phase shifting mechanisms. The voltage bias may be selected to adjust the phase shift mechanism so to alter an effective index of a waveguide coupled to the phase shifting mechanism. By altering the effective index of the waveguide, a resonator structure formed of the waveguide can be tuned to a distinct wavelength of the m optical signals. To implement entries of the weight matrices854, each phase shifting mechanism can be detuned according to the weight of each entry by adjusting the voltage bias accordingly.

Accordingly, input vector852can be multiplied by different sets of weights to obtain a number of outputs. For example, the output signals from the weight matrix core820propagate to a detection layer840for determining entries of each output vector856a-856nbased on amplitudes of the output signals at each wavelength. In the example shown inFIG.8A, detection layer840comprises a plurality of demultiplexers842a-842ncoupled to a subset of outputs of the waveguides822a-822n. The number of demultiplexers842a-842nmay be half of the number of waveguides822a-822n(e.g., half of the number of entries in the input vector852). Each of the demultiplexers842a-842ncan be configured to filter each wavelength onto separate output waveguides coupled thereto. The demultiplexers842a-842nmay be provided as coarse wavelength division multiplexing (CWDM) demultiplexers that can be implemented as de-interleavers, contra-directional couplers, or the like. Each of the demultiplexers842a-842ncan be operated to separate output signals from into individual output waveguides according to different wavelengths. For example, demultiplexers842a-842nreceive weighted signals from weight matrix core820and separate (e.g., filter) each wavelength of light onto a distinct output waveguide. Thus, each optical signal on each output waveguide is indicative of an entry of an output vectors856that corresponds to each wavelength.

Detection layer840also comprises photodetectors844a-1through844m-ncoupled to the demultiplexers842a-842n. Photodetectors844a-1through844m-ndetect the optical signals on a respective output waveguide, where the amplitude of each detected signal is representative of an entry of output vectors856. For example, demultiplexer842ais coupled to a first subset of photodetectors844a-1through844m-n. As described above, demultiplexer842afilters the output from weight matrix core820into distinct wavelengths of light, each of which is supplied to one of the photodetectors844a-1through844m-nvia a corresponding output waveguide. Thus, photodetector844a-1detects an amplitude of light at a first wavelength that is representative of a first entry of output vector856a, while photodetectors844b-1and844m-1detect an amplitude of light at a second and an mthwavelength that is representative of the first entries of output vector856band output vector856n, respectively. Similarly, photodetectors844a-844ndetect an amplitude of light at the first wavelength that is representative of a nthentry of output vector856a, while photodetectors844b-nand844m-ndetect an amplitude of light at the second and the mthwavelength that is representative of the nthentries of output vector856band output vector856n, respectively.

Accordingly, optical system800can used to multiple a plurality of weight matrices854to an input vector852in parallel (e.g., simultaneously) to determine a plurality of output vectors856. Thus, an n-port device (e.g. an optical system800having an n number of input ports) that can perform any n×n unitary linear transformation can by arranging multiple unit cells824a-824nin various mesh configurations. Since the resonator structures of the unit cells824a-824nare narrowband devices, by tuning signal mixers and phase tuning components, different weight matrices can be constructed for each port by leverage multiple wavelengths, as described above. Thus, multiple MVM operations can be performed simultaneously, thereby reducing computation time.

In example implementations, optical system800may be utilized for convolutional neural network (CNN) applications. For example, a feature detector implemented as a CNN may include a two-dimensional (2-D) array of weights (e.g., a matrix of weights). A grayscale image used as an input may have a single color channel that defines entries of the input vector852. Multiple filters for detecting multiple features can be implemented as weights in a weight matrix. Features may be, for example but not limited to, edges, curves, shapes, etc. Thus, each filter may be implemented as a 2-D array of weights, where each 2-D array of weights is implemented as a single weight matrix of weight matrices854. Each filter can then be applied to the input image using optical system800to perform dot products of the input image (e.g., input pixels values) and the multiple filters (e.g., weight matrices854). Conventionally MZI based ONNs implemented for the above may process one filter at a time. Whereas, the implementations disclosed herein can obtain results for multiple multiplication operations in parallel, which enhances the computation density and reduces computation time.

As alluded to above,FIG.9Adepicts an optical system900that can be implemented to perform MVM on a set of input vectors.FIG.9Bdepicts the operation950, whereby a set of m weight matrices954is applied to a set of m input vectors952to produce an output vector956, where m is an integer greater than one. The weight matrices954comprises a set of m weight matrices954a-954nand the set of m input vectors comprises input vectors952a-952n.

To achieve multiple MVMs operation950, optical system900comprises input layer930that receives an input signal from an optical source (such as a comb laser or other light source as known in the art). The optical source may emit a plurality of input signals932a-932nonto a plurality of waveguides922a-922n. Each source signal932a-932nmay comprises a number m of optical signals, each propagating at a distinct wavelength. The number of waveguides922a-922nmay correspond to the number n of entries in an input vector952and the number of optical signals at distinct wavelengths may correspond to the number m of weight matrices954a-954n.

The input layer930comprises a plurality of narrowband modulators934a-1through934m-n, each configured to modulate an amplitude of the input signals932a-932nat a distinct wavelength according to entries of the input vector952. By modulating the input signals932a-932n, entries of the input vectors952can be encoded onto the input signals. For example, each input vector952a-952nmay be associated with a distinct wavelength of light (e.g., input vector952amay correspond to a first wavelength, input vector952bmay correspond to a second wavelength, and input vector952nmay correspond to an mthwavelength). A first subset934aof narrowband modulators may be configured to modulate light of a first wavelength on each of the source signals932a-932naccording to entries of input vector952a. A second subset934bof narrowband modulators may be configured to modulate light of a first wavelength on each of the source signals932a-932naccording to entries of input vector952b. An mthsubset934mof narrowband modulators may be configured to modulate light of an mthwavelength on each of the source signals932a-932naccording to entries of input vector952n.

Each waveguide922a-922ncorresponds to each row of the input vectors952a-952n, such that a given narrowband modulator coupled to a given waveguide modulates an optical signal according to an entry of the row of an input matrix corresponding to the given waveguide, where the modulated optical signal propagates at a wavelength corresponding to the input matrix. That is, for example, narrowband modulate934a-1modulates an optical signal at the first wavelength according to a first entry of input vector952a, narrowband modulate934b-1modulates an optical signal at the second wavelength according to a first entry of input vector952b, and narrowband modulate934m-1modulates an optical signal at the mthwavelength according to a first entry of input vector952n. Similarly, narrowband modulate934a-nmodulates an optical signal at the first wavelength according to a nthentry of input vector952a, narrowband modulate934b-nmodulates an optical signal at the second wavelength according to a nthentry of input vector952b, and narrowband modulate934m-nmodulates an optical signal at the mthwavelength according to an nthentry of input vector952n. Thus, entries of the input vectors952can be encoded onto the optical signals divided into separate wavelength channels.

Optical system900can comprise a control circuit (not shown inFIG.9A), such as control circuit860ofFIG.8A. The control circuit can be utilized to control the modulators934a-1through934m-naccording to the entries of the input vectors952.

The encoded input signals are supplied to a weight matrix core920(also referred to as a cross bar array). Weight matrix core920may be substantially similar to weight matrix core820. As such, weight matrix core920comprises a plurality of unit cells924a-924n, each of which may be implemented as any of unit cell100, unit cell200, unit cell500, or unit cell600. Each unit cell924a-924ncan be tuned according to entries in the weight matrices954. For example, each unit cell924a-924nmay comprise a number m of sub-unit cells, as described above in connection withFIGS.1,2,5, and6. Each sub-unit cell, which comprises one or more resonant structures as described above, may be tuned for a distinct wavelength of the m optical signals. Signal mixing components and phase tuning components of sub-unit cell can then be detuned so to apply weights according to entries of the weight matrices954. As discussed above, the control circuit can issue control commands for tuning the unit cells924a-924naccording to the entries of the weight matrices954.

Thus, a set of input matrices952can be multiplied by a set of weights to obtain output vector956. For example, the weighted output signals from the weight matrix core920propagate to a detection layer940for determining entries of output vector956based on amplitudes of the weighted output signals detected by a plurality of photodetectors944a-944n. Photodetectors944athrough944ndetect the optical signals on a respective waveguide. The optical signals on each waveguide are accumulated at each of the photodetectors944a-944nand the amplitude detected by each the photodetectors944a-944nis representative of a respective entry of output vector956.

According to an example, the maximum number of unit cells to build a N-dimension unitary matrix may be (N−1)*(N/2) unit cells. In an example implementation, an input image may be color image, which can be represented as three input vectors (e.g., input vectors952) corresponding to the red, green, and blue (RGB) channels of the input image. The weight matrix (e.g., weight matrices954) may represent weights for each color channel. The weight matrices954can be multiplied with each input vector952, and a summation (e.g., total amplitude on each waveguide) of each output detected at the detection layer940provides entries for output vector956.

For a conventional RGB image, the raw image may include three color channels. A convolutional layer of a neural network can be provided to compute the output of neurons that are connected to local regions in the input, where each computation is a dot product between the weights and the local region of the input image. The weight matrix for each color channel may be unique, and thus implemented as a set of three weight matrices. Each entry in the output vector can be computed by elementwise multiplication of each color channel of a region of the input with a corresponding filter (e.g., weight matrix) and summing up the multiplication. According to this illustrative example, optical system900can be configured to achieve all RGB channels convolution within one computation cycle by performing all three color channel computations in parallel. Thus, computation efficiency can be improved by three times compared to conventional MZI-based ONNs.

As alluded to above,FIG.10Adepicts an optical system1000that can be implemented to perform GEMM.FIG.10Bdepicts the operation1050, whereby a set of m weight matrices1054is applied to a set of m input matrices1052to produce a set of m output matrices1056, where m is an integer greater than zero. In the example ofFIG.10B, the set of m weight matrices1054comprises weight matrices1054a-1054nand the set of m input vectors comprises input matrices1052a-1052n. Thus, optical system1000may be implemented to execute a single GEMM operation (e.g., m=1) or a plurality of GEMM operations (e.g., m>1) in parallel.

In this case, optical system1000leverages WDM and multi-FSRs for encoding input matrices1052into input signals that can be supplied into a resonator-based weight matrix core1020. By utilizing WDM and multi-FSRs, optical system1000can execute GEMM through parallel photonic processing. Weight matrix core1020can be implemented using a plurality of unit cells, such as unit cells100,200,500, and/or600discussed above. Weight matrix core1020can be encoded with weight matrices1054, such that when set of input matrices1052is input into weight matrix core1020, set of input matrices1052is multiplied with weight matrices1054to produce output matrices1056.

In the illustrative example ofFIG.10A, optical system1000comprises input layer1030that receives an input signal from an optical source (such as a comb laser or other light source as known in the art). The input signals (shown inFIG.10as signals1032athrough1032n) each can comprise a plurality of optical signals at a plurality wavelength (e.g., an initial resonance wavelength and at least ±(k−1) resonance wavelengths corresponding to multiple FSRs of resonator structures included in the weight matrix core1020). For example,FIG.10Adepicts a first FSR group (FSR1) through an nthFSR group (FSRn), where the number of FSRs corresponds to a n number of rows in the set of input matrices1052(e.g., number of rows in input matrix1052a,1052b, or1052n). Each FSR group comprises an m number of wavelengths corresponding to the number m of input matrices that make up set of input matrices1052. The first FSR group may correspond to initial resonance wavelengths of resonator structures including the sub-unit cells of weight matrix core1020, while a second through nthFSR groups correspond to a number n of FSRs of the resonator structure.

Entries of set of input matrices1052can be encoded into the plurality of optical signals of input signals1032a-1032nand the encoded optical signals can be supplied to waveguides1022a-1022das inputs into weight matrix core1020. For example, input layer1030comprises a plurality of narrowband modulators that are configured to encode each optical signal with an entry from set of input matrices1052. For example, input layer1030includes a plurality of narrowband modulator groups1033a-1033n, each corresponding to an FRS group (e.g., narrowband modulator group1033acorresponds to FSR1 and narrowband modulator group1033ncorresponds to FSRn). Each narrowband modulator group comprises a plurality of sub-groups1034a-1through1034m-nof narrowband modulators. Each sub-group of narrowband modulators comprises a number n of narrowband modulators configured to modulate an optical signal at a wavelength. For example, narrowband modulators of sub-group1034a-1are configured to modulate optical signals at a first wavelength and narrowband modulators of sub-group1034m-1are configured to modulate optical signals at an mthwavelength, where the first and mthwavelengths are part of FSR1. Whereas, narrowband modulators of sub-group1034a-nare configured to modulate optical signals at an nthresonance wavelength of the first wavelength and narrowband modulators of sub-group1034m-nare configured to modulate optical signals at the nthresonance wavelength of the mthwavelength (e.g., FSRn).

Accordingly, entries of each input matrix1052a-1052ncan be encoded into a plurality of optical signals my modulating the optical signals by splitting the signals by both wavelength and FSR groups. For example, each waveguide1022a-1022ncorresponds to a column of the input matrices1052a-1052nand each FSR group corresponds to a row. Additionally, each wavelength corresponds to an input matrix1052a-1052n. As such, each narrowband modulator can be controlled to modulate the optical signals according to an entry of input matrix1052corresponding to the narrowband modulator.

The encoded optical signals are supplied to the weight matrix core1020as input signals. The weight matrix core1020may be substantially similar to weight matrix core1020, and comprises a plurality of unit cells1024a-1024nimplemented as any of unit cell100, unit cell200, unit cell500, or unit cell600. Each unit cell1024a-1024ncan be tuned according to entries in the weight matrices1054. For example, each unit cell1024a-1024nmay comprise a number m of sub-unit cells, as described above in connection withFIGS.1,2,5, and6. Each sub-unit cell, which comprises one or more resonant structures as described above, may be tuned for a distinct wavelength of the m optical signals. As such, resonator structures can operate on at resonance frequency and n number of FSRs. Signal mixing components and phase tuning components of sub-unit cell can then be detuned so to apply weights according to entries of the weight matrices1054.

Thus, a set of input matrices1052can be multiplied by a set of weights to produce a set of output matrices1056. For example, the weighted output signals from the weight matrix core1020propagate to a detection layer1040for determining entries of output matrices1056based on amplitudes of the weighted output signals detected by a plurality of photodetectors1044a-1044n. The output from the weight matrix core1020is supplied to a set of demultiplexers1042a-1042n, provided as coarse wavelength division multiplexing (CWDM) demultiplexers that can be implemented as de-interleavers, contra-directional couplers, or the like. Demultiplexers1042a-1042ncan be operated to separate output signals from into individual output waveguides according to different FSR groups. Once separated (e.g., filtered) according to FSR groups, each FSR group can be separated (e.g., filtered) into individual output waveguides according to wavelength included in the respective FSR group (e.g., as shown by demultiplexer1042bas an example). Photodetectors1044athrough1044ndetect the optical signals for each wavelength on a respective output waveguide. The optical signals on each output waveguide are accumulated at each of the photodetectors1044a-1044nand the amplitude detected by each the photodetectors1044a-1044nis representative of a respective entry of output matrices1056.

Accordingly, by exploiting multiple FSRs of the resonator structures, MVMs of a single photonic core can be extended into GEMM operations as discussed above. In the illustrative example ofFIG.10B, a weight matrices1054is provided as a 3×n×n tensor that multiplies with a 3×n×k tensor as set of input matrices1052, which results in a 3×n×k tensor as output matrices1056. As discussed above, due to the periodicity of the resonances, each resonator structure in the weight matrix core1020has resonances at an initial wavelength (λ1) and one or more resonance wavelengths corresponding to multiple FSRs resonance (e.g., λ1+Δλ, . . . , λ1+(k−1)Δλ, where Δλ is the FSR). By tuning the resonances of the resonator structures, an amount of optical power on each waveguide can be dropped, which multiplies a weight to the optical signal on the waveguide. The response on different FSRs by a resonant structure may be substantively similar, thereby enabling the same weight value to be applied to each resonant wavelength. As discussed above, the input into the weight matrix core1020includes vector {x11, . . . , xn1} in column 1 of input matrices1052encoded by all the WDM wavelength channels λ1, λ1+Δλ, . . . , λ1+(n−1)Δλ at corresponding input waveguide1022a-1022n. The depth of the input volumes (e.g., input matrix1052aand input matrix1052b) can be parallelly encoded by WDM wavelength channels λ1, λ2and λ3. The encoded input signals are then weighted as discussed above and selected by demultiplexers1042a-1042nat each output port. As a result, by encoding the k columns of X into k FSR groups, encoding the3depth channels of set of input matrices1052in 3 wavelengths, and detecting the different signals at each output port, MVMs between weight matrices1054and set of input matrices1052, e.g., parallelization of GEMM between set of input matrices1052and weight matrices1054, can be implemented in a single device (e.g., optical system1000).

FIG.11illustrates an example computing component that may be used to implement an ONN in accordance with various embodiments. Referring now toFIG.11, computing component1100may be, for example, a server computer, a controller, or any other similar computing component capable of processing data. In the example implementation ofFIG.11, the computing component1100includes a hardware processor1102, and machine-readable storage medium for1104.

Hardware processor1102may be one or more central processing units (CPUs), semiconductor-based microprocessors, and/or other hardware devices suitable for retrieval and execution of instructions stored in machine-readable storage medium1104. Hardware processor1102may fetch, decode, and execute instructions, such as instructions1106-1114, to control processes or operations for ONNs. As an alternative or in addition to retrieving and executing instructions, hardware processor1102may include one or more electronic circuits that include electronic components for performing the functionality of one or more instructions, such as a field programmable gate array (FPGA), application specific integrated circuit (ASIC), or other electronic circuits.

Hardware processor1102may execute instruction1106to tune resonances of a plurality of resonator structures of a weight matrix core to a plurality of resonance wavelengths. For example, as described above in connection withFIGS.8A-10B, a weight matrix core may be provided having a plurality of unit cells, such as such as, unit cells100ofFIG.1, unit cell200ofFIG.2, unit cell500ofFIG.5, and/or unit cell600ofFIG.6. Each unit cell comprises one or more sub-unit cells comprising one or more resonator structures. A bias (e.g., voltage bias) can be applied to phase shift mechanism, such as phase shifting mechanisms226,228, and/or234, coupled to the resonator structures to tune the resonator structure to a desired resonance frequency (e.g., resonance wavelength).

Hardware processor1102may execute instruction1108to adjust the resonances of the plurality of resonator structures to detune the resonances relative to the plurality of resonance wavelengths according to entries of a plurality of weight matrices. Each of the plurality of weight matrices may be associated with a resonance wavelength of the plurality of resonance wavelengths. For example, as described, as described above in connection withFIGS.8A-10B, a plurality of weight matrices may be provided that each comprise entries of learned weights. Each weight matrix may be associated with a resonance wavelength set by the tuning at instructions1106, such that each unit cell comprises a sub-unit cell that corresponds to a weight matrix. Based on the entries each weigh matrix, the resonance frequencies of the resonator structures of each unit cell can be detuned to act on optical signals encoded with input data according to the weights of the corresponding weight matrix.

Hardware processor1102may execute instruction1110to input a plurality of first optical signals at the plurality of resonance wavelengths onto a plurality of waveguides, each first optical signal encoded with information according to one or more input matrices. The plurality of resonator structures are optically coupled between adjacent waveguides of the plurality of waveguides into which the plurality of first optical signals are supplied. Example implementations of the first optical signals are provided above in connection withFIGS.8A-10B, such that the implementations disclosed herein can be used to perform various multiplication processes on various matrices. The plurality of first optical signals can be encoded with data representing entries of a single matrix or vector to perform multiple matrix-vector multiplications on a single input (e.g.,FIGS.8A and8B). As another example, the plurality of first optical signals can be encoded with data representing entries of a plurality of vectors to perform matrix-vector multiplication operations on the multiple inputs (e.g.,FIGS.9A and9B). As yet another example, the plurality of first optical signals can be encoded with data representing entries of a set of matrices for performing multiple general matrix multiplication operations (e.g.,FIGS.10A and10B).

Hardware processor1102may execute instruction1112to detect optical power output from the plurality of resonator structures. For example, as described in connection withFIGS.8A-8B, outputs from the weight matrix core can be demultiplexed into separate outputs according to resonance frequency and an optical power detected for each output.

Hardware processor1102may execute instruction1114to generate one or more entries of an output matrix based on the detected optical power. For example, as described in connection withFIGS.8A-10B, the detected optical power from instructions1112is representative of an entry in a resultant matrix, which can be used to construct an output of the multiplication operations, as described above.

FIG.12depicts a block diagram of an example computer system1200in which various of the embodiments described herein may be implemented. The computer system1200includes a bus1202or other communication mechanism for communicating information, one or more hardware processors1204coupled with bus1202for processing information. Hardware processor(s)1204may be, for example, one or more general purpose microprocessors.

The computer system1200also includes a main memory1206, such as a random access memory (RAM), cache and/or other dynamic storage devices, coupled to bus1202for storing information and instructions to be executed by processor1204. Main memory1206also may be used for storing temporary variables or other intermediate information during execution of instructions to be executed by processor1204. Such instructions, when stored in storage media accessible to processor1204, render computer system1200into a special-purpose machine that is customized to perform the operations specified in the instructions.

The computer system1200further includes a read-only memory (ROM)1208or other static storage device coupled to bus1202for storing static information and instructions for processor1204. A storage device1210, such as a magnetic disk, optical disk, or USB thumb drive (Flash drive), etc., is provided and coupled to bus1202for storing information and instructions.

The term “non-transitory media,” and similar terms, as used herein refers to any media that store data and/or instructions that cause a machine to operate in a specific fashion. Such non-transitory media may comprise non-volatile media and/or volatile media. Non-volatile media includes, for example, optical or magnetic disks, such as storage device1210. Volatile media includes dynamic memory, such as main memory1206. Common forms of non-transitory media include, for example, a floppy disk, a flexible disk, hard disk, solid-state drive, magnetic tape, or any other magnetic data storage medium, a CD-ROM, any other optical data storage medium, any physical medium with patterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, NVRAM, any other memory chip or cartridge, and networked versions of the same.

The computer system1200also includes a communication interface1218coupled to bus1202. Communication interface1218provides a two-way data communication coupling to one or more network links that are connected to one or more local networks. For example, communication interface1218may be an integrated services digital network (ISDN) card, cable modem, satellite modem, or a modem to provide a data communication connection to a corresponding type of telephone line. As another example, communication interface1218may be a local area network (LAN) card to provide a data communication connection to a compatible LAN (or WAN component to communicated with a WAN). Wireless links may also be implemented. In any such implementation, communication interface1218sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information.

The computer system1200can send messages and receive data, including program code, through the network(s), network link and communication interface1218. In the Internet example, a server might transmit a requested code for an application program through the Internet, the ISP, the local network and the communication interface1218.

The received code may be executed by processor1204as it is received, and/or stored in storage device1210, or other non-volatile storage for later execution.