HETEROGENEOUSLY INTEGRATED OPTICAL NEURAL NETWORK ACCELERATOR

Embodiments of the present disclosure are directed toward techniques and configurations for an optical accelerator including a photonics integrated circuit (PIC) for an optical neural network (ONN). In embodiments, an optical accelerator package includes the PIC and an electronics integrated circuit (EIC) that is heterogeneously integrated into the optical accelerator package to proximally provide pre- and post-processing of optical signal inputs and optical signal outputs provided to and received from an optical matrix multiplier of the PIC. In some embodiments, the EIC is a single EIC or discrete EICs to provide pre- and post-processing of the optical signal inputs and optical signal outputs including optical to electrical and electrical to optical transduction. Other embodiments may be described and/or claimed.

FIELD

Embodiments of the present disclosure generally relate to the field of optoelectronics and optical neural network processors, and more particularly, to techniques and configurations for providing integrated silicon photonics optical devices.

BACKGROUND

Artificial neural networks (ANNs) are computing systems vaguely inspired by the brain. Conventional ANNs typically rely on electronic components or architectures based on CMOS-related technology. An optical neural network (ONN) is a physical implementation of an artificial neural network which includes optical components. Applications that may require fast processing of high amounts of data, such as voice recognition, image processing, and search rankings, are fed from a high-performance CPU for processing by the ONN. Recently, ONN accelerators built with discrete optical and electrical components have begun to emerge. Relative to their predecessors, the ONN accelerators can reach higher power efficiency, e.g., more than tens of Tera-Operations/Second per Watt (TOPS/W), faster computation speeds, e.g., clock frequencies higher than 10 Giga-Hertz (GHz), as well as lower latency, e.g. less than 1 nanosecond (ns).

DETAILED DESCRIPTION

Embodiments of the present disclosure describe techniques and configurations for an apparatus for an optical neural network (ONN). In embodiments, the apparatus includes e.g., a heterogeneously integrated optical accelerator including a stacked photonics integrated circuit (PIC) and an electronics integrated circuit (EIC). In embodiments, the PIC includes an ONN having one or more layers of optical unitary matrix multipliers and an optical nonlinearity function implemented via nonlinear optical devices. In embodiments, the EIC is stacked in a manner vertically above or below the PIC in a single optical accelerator package with the PIC to proximally provide pre- and post-processing of optical signal inputs and optical signal outputs including optical to electrical and electrical to optical transduction. Integration of the EIC into the optical accelerator package as described may result in higher bandwidth, higher density and lower power consumption due to a proximal location of radiofrequency (RF) interfaces of the PIC and EIC.

In embodiments, the optical signal inputs and optical signal outputs are provided by the optical accelerator to (and received from) a CPU, such as, e.g., a server CPU (e.g., Intel XEON™ or other high performance CPU). In some embodiments, the optical accelerator is considered a co-processor to the CPU, which may be located on a motherboard external to the optical accelerator package. In other embodiments, the CPU is integrated in the optical accelerator package with the PIC and the EIC. In some embodiments, the EIC is a single integrated EIC including some or substantially all functions required for pre-and post-processing of data provided between the PIC die and the CPU. In some embodiments, the EIC includes a plurality of integrated EICs or discrete EIC dies that integrate single or multiple functions of the same. In embodiments, the optical unitary matrix multiplier comprises a plurality of 2×2 unitary optical matrices optically interconnected, and each 2×2 unitary optical matrix comprises a plurality of phase shifters to phase shift, split, or combine one or more of the optical signal inputs.

The description may use the phrases “in an embodiment,” or “in embodiments,” which may each refer to one or more of the same or different embodiments. Furthermore, the terms “comprising,” “including,” “having,” and the like, as used with respect to embodiments of the present disclosure, are synonymous. The term “coupled with,” along with its derivatives, may be used herein. “Coupled” may mean one or more of the following. “Coupled” may mean that two or more elements are in direct physical or electrical contact. However, “coupled” may also mean that two or more elements indirectly contact each other, but yet still cooperate or interact with each other, and may mean that one or more other elements are coupled or connected between the elements that are said to be coupled with each other. The term “directly coupled” may mean that two or more elements are in direct contact.

As noted above, stacked photonics integrated circuit (PIC) of the optical accelerator includes a plurality of 2×2 unitary optical matrices optically interconnected. In embodiments these 2×2 unitary optical matrices include 2×2 unitary directional optical couplers and 2×2 unitary MMI optical couplers are described and shown in connection withFIGS. 1-9below. For example, in various embodiments, the matrix multipliers include a plurality of 2×2 unitary adiabatic directional optical couplers such as the 2×2 unitary adiabatic directional optical coupler ofFIG. 2, 2×2 unitary directional optical couplers and adiabatic directional optical couplers having one or more common or differential phase shifters ofFIG. 3, or 2×2 unitary multi-mode interference (MMI) optical couplers having one or more of differential phase shifters and/or common phase shifters ofFIG. 5. Note thatFIGS. 6-8illustrate side views and cross-sectional views of various embodiments of the devices introduced inFIGS. 1-5.

FIG. 1is illustrates an example top view of a 2×2 unitary directional optical coupler100(also referred to as “directional optical coupler100”), in accordance with embodiments. In embodiments, a configuration of directional optical coupler100allows for a 2×2 optical unitary matrix multiplier that is able to perform a 2×2 unitary linear transformation on optical signals in a limited or compact space. As shown, directional optical coupler100includes a first optical waveguide101and a second optical waveguide103. First optical waveguide101and second optical waveguide103are coupled to form a 2×2 optical unitary matrix to receive a respective first input optical signal (e.g., E1, in) and a second input optical signal (e.g., E2, in). As seen fromFIG. 1, optical waveguide101and103form a respective first arm and a second arm that diverge at a first end (e.g.,116) and a second end (e.g.,118) and converge along a middle portion of a path (e.g., path115). In embodiments, path115runs along first optical waveguide101and second optical waveguide103in a substantially parallel manner. In the embodiment, path115includes or integrates a plurality of phase shifters, (e.g., phase shifter107and phase shifter109) to assist in transforming the first optical signal or the second optical signal into a first output optical signal (e.g., E1 out) and second output optical signal (e.g., E2 out) to be output from the 2×2 optical unitary matrix. In embodiments, the transformation includes a combining, splitting, and phase shifting of the first input optical signal and the second input optical signal.

As will be discussed further, in embodiments, phase shifters107and109include at least one of an electro-optical induced index modulator, thermal-optics induced index modulator, image-spot modulator, or opto-electronic-mechanical modulator, to allow for tunable power at output waveguides. In the embodiment shown, phase shifter107applies a first phase shift ø and phase shifter109applies a second phase shift Θ. As noted previously, in embodiments, directional optical coupler100performs a linear unitary transformation via matrix multiplication to input optical signals E1,inand E2, in. For example, the transfer matrix for the directional optical coupler ofFIG. 1can be expressed as:

Note that in embodiments, path115has a length of or includes a critical coupling length, l, to allow the unitary transformation of optical signals in optical waveguide101and103. Thus, in the embodiment, 2×2 unitary directional optical coupler100includes phase shifters107and109, which may also serve as optical splitters and optical combiners integrated along the critical coupling length l, to respectively split or combine the first input optical signal and/or second input optical signal. In embodiments, critical coupling length l is determined to be a length to, in combination with a width of gap108, promote or allow the first optical signal to switch from first optical waveguide101to the second optical waveguide103or vice-versa. Thus, tuning of one or more of the phase shifters causes the first input optical signal or the second input optical signal (or a portion thereof) to be switched into either of the arms to effectively form an analog switch.

As noted above inFIG. 1, optical waveguide101and103form a respective first arm and a second arm that diverge at a first end (e.g.,116) and a second end (e.g.,118) and converge along a middle portion of a path (e.g., path115). In embodiments, path115is a substantially parallel path along first optical waveguide101and second optical waveguide103. Furthermore, note that path115includes a gap108, having a width w, which runs between first optical waveguide101and second optical waveguide103along the substantially parallel path. In embodiments, the configuration of the 2×2 optical unitary matrix including the first arm and the second arm that converge to at least a critical coupling length l and gap108allow for the matrix multiplication to be performed in a limited or compact space.

Referring now to the embodiment ofFIG. 2, which illustrates an example top view of a 2×2 unitary adiabatic directional optical coupler200(also sometimes referred to as “adiabatic directional coupler”). InFIG. 2, adiabatic directional optical coupler200includes a first optical waveguide121and second optical waveguide123evanescently coupled to form a 2×2 optical unitary matrix. In embodiments, adiabatic directional optical coupler200, however, is formed to operate without optical loss or substantially any optical loss. In the embodiments shown, adiabatic directional optical coupler200is formed to include optical waveguides that have dissimilar widths, core dimensions, or bend diameters, from each other and/or that vary in their widths or diameters along a length of an optical path that includes a plurality of phase shifters, e.g., phase shifter132and134. In the embodiment, adiabatic directional optical coupler200receives a respective first input optical signal (e.g., E1,in) and a second input optical signal (e.g., E2, in) and outputs a respective first output optical signal (e.g., E1 out) and second output optical signal (e.g., E2 out). As shown, optical waveguide121and optical waveguide123converge to run alongside each other to direct the first input optical signal and the second input optical signal along optical path225(“path225”). In embodiments, path225may include a critical coupling length, l, that may be longer or shorter than path225, but that promotes adiabatic evanescent coupling between optical signals in optical waveguide121and123.

As noted above and as shown inFIG. 2, first optical waveguide121has a different width, core dimension, or bend diameter, from second optical waveguide123. Furthermore, in some embodiments, the width of one or more of first optical waveguide121and second optical waveguide123varies along path225. Accordingly, adiabatic directional optical coupler200includes a first optical waveguide121separated from a second optical waveguide123by a gap208. In embodiments, gap208varies in width along path225due to varying width of first optical waveguide121or second optical waveguide123. In embodiments, gap208includes a width that in addition to a critical coupling length l, is determined to promote evanescent coupling (e.g., at136) between a first input optical signal and second input optical in first optical waveguide121and second optical waveguide123.

As seen inFIG. 2, optical waveguides121and123form a respective first arm and a second arm that diverge at a first end (e.g.,126) and a second end (e.g.,128) and converge along a middle portion of a substantially parallel path (e.g., path225). Note optical waveguides121and123form a concave up or concave down shape. Note that as shown and discussed in connection withFIGS. 3 and 6below, it is understood that a type and number of phase shifters in directional optical coupler100and adiabatic directional optical coupler200will vary.

FIG. 3illustrates an example top view of a plurality of 2×2 unitary directional optical couplers and adiabatic directional optical couplers including one or more common or differential phase shifters, in accordance with embodiments. On a left side ofFIG. 3, directional coupler100and adiabatic directional coupler200as described above inFIGS. 1 and 2are reproduced. Note that directional coupler100and adiabatic directional coupler200include differential phase shifters. For example, unitary directional optical coupler100includes phase shifter107, which applies a phase shift ø, and phase shifter109, which applies a phase shift Θ, to apply a differential phase shift (e.g., phase shift ø−phase shift Θ). Similarly, adiabatic directional coupler200includes phase shifters132and phase shifter134to apply a differential phase shift (phase shift ø−phase shift Θ) to a first input optical signal (e.g., E1,in) and a second input optical signal (e.g., E2, in) of adiabatic directional coupler200.

In contrast, directional optical coupler304and adiabatic directional optical coupler308on a right side ofFIG. 3include both differential phase shifters and a common or single phase shifter that is common to both optical waveguides. As shown, directional optical coupler304includes a first optical waveguide330and a second optical waveguide333. Common phase shifter315is located or integrated on a path common to each of first optical waveguide330and second optical waveguide333. In contrast, external phase shifters317and319are located on paths335and337that are external to a path325that integrates common phase shifter315, which implements a unitary transformation of the 2×2 unitary matrix. In the example embodiment, external phase shifters317and319of directional optical coupler304together apply a differential phase shift of phase shift Θ1−phase shift Θ2.

Similarly, in embodiments, adiabatic directional coupler308includes a first optical waveguide351and a second optical waveguide353including a common phase shifter322. Common phase shifter322is located or integrated on a path common to each of first optical waveguide351and second optical waveguide353. In contrast, external phase shifters325and327are located on paths355and357that are external to a path365that integrates common phase shifter322, which implements a unitary transformation. In embodiments, external phase shifter325applies phase shift Θ1while external phase shifter327applies a phase shift of Θ2to together apply a differential phase shift of Θ1−Θ2.

Referring now toFIG. 4, which illustrates a top view of two example 2×2 unitary multi-mode interference (MMI) optical couplers, in accordance with embodiments. InFIG. 4, each of unitary MMI optical coupler400and a unitary MMI optical coupler403include respective multi-mode (MMI) waveguide structures410and420that intersect an optical path. In embodiments, the MMI waveguide structures are formed such that modes of a first optical signal and modes of a second optical signal interfere with each other to assist in performing a unitary transformation of input optical signals. Note that unitary MMI optical coupler400and unitary MMI optical coupler403are similar to each other, with the exception of a differing shape of a bowed shape of MMI waveguide structure420of unitary MMI optical coupler403.

As shown, unitary MMI optical coupler400includes a first optical waveguide401and a second optical waveguide403coupled to form a 2×2 optical unitary matrix to receive a respective first input optical signal (e.g., E1 in) and a second input optical signal (e.g., E2 in). In embodiments, MMI waveguide structure407has a length Lπ and a width We. Optical waveguide401and optical waveguide403run alongside each other to direct the first input optical signal and the second input optical signal along an optical path425that intersects with MMI waveguide structure410for length Lπ. In the embodiment, optical path425includes or integrates a plurality of phase shifters to assist in performing a unitary transformation of the first optical signal and/or the second optical signal into a first output optical signal (e.g., E1out) and second output optical signal (e.g., E2 out). In the embodiment, MMI optical coupler400includes phase shifter407, phase shifter408, and phase shifter409along length Lπ.

Similarly, unitary MMI optical coupler403includes a first optical waveguide421and a second optical waveguide423coupled to form a 2×2 optical unitary matrix to receive a respective first input optical signal (e.g., E1 in) and a second input optical signal (e.g., E2 in). In the embodiment, optical path426includes or integrates a plurality of phase shifters to assist in performing a unitary transformation of the first optical signal or the second optical signal into a first output optical signal (e.g., E1out) and second output optical signal (e.g., E2out) to be output from the 2×2 optical unitary matrix. In the embodiment, MMI optical coupler403includes phase shifter447, phase shifter441, and phase shifter449along length Lπ.

In embodiments, MMI waveguide structure420has a length Lπ and a width We. Optical waveguide421and optical waveguide423run alongside each other to direct the first input optical signal and the second input optical signal along an optical path426that intersects with MMI waveguide structure420for length Lπ. As noted above, MMI waveguide structure420has a differing shape than MMI waveguide structure410. In the embodiment shown, MMI waveguide structure420has a curved or bowed shape along lengthwise perimeters451and453. In embodiments, the curved or bowed shape provides additional space to allow interference of the modes of the first optical input signal and a second optical input signal.

Note that, in embodiments, length Lπ of MMI optical couplers400and403includes a fraction or a multiple of a critical beating length Lc of the two lowest order modes, with a multiple of a phase shifter combination for optimal phase shift efficiency. For example, if width Weis a width of MMI optical couplers400or403, βo is the propagation foundation of the foundational mode, β1is the propagation constant of a first order mode, nris the effective refractive index of an optical waveguide, e.g., MMI waveguide structure407or420, and λo is the wavelength of the light, then:

Note that, although MMI optical coupler400and403each include three phase shifters, it is understood that in other embodiments, the MMI optical couplers include any suitable number of phase shifters or arrangements of phase shifters to phase shift the first input optical signal and/or the second input optical signal to perform a unitary transformation. In some examples, MIMI optical couplers includes successive phase shifters along the optical path that includes length Lπ. In some examples, the MMI optical couplers also include a combination of both common phase shifters and differential phase shifters as will be shown inFIG. 5. In embodiments, modes of the first optical signal and the second optical signal interfere in the MM waveguide to output an optical signal at a power ratio that can be adjusted according to unitary matrix algebra.

FIG. 5illustrates a top view of example 2×2 unitary multi-mode interference (MMI) optical couplers, having differential phase shifters and/or common phase shifters. Unitary MMI optical couplers400and403ofFIG. 4, whose elements were shown and described in connection withFIG. 4, are reproduced on a left column ofFIG. 4. Thus, unitary MMI optical coupler400includes phase shifter407and phase shifter409to apply a differential phase shift (e.g., phase shift ø1−phase shift ø2). Similarly, MMI optical coupler403, having curved MMI waveguide structure420, includes phase shifters447and449to apply a differential phase shift (phase shift ø1−phase shift ø2) on its respective first optical waveguide and second optical waveguide. Each of MMI optical coupler400and403also include respective phase shifters408and441to apply a phase shift Θ.

Unitary MMI optical couplers504and508on a right side ofFIG. 5include elements similar to or the same as unitary MMI optical couplers400and403. In contrast to unitary MMI optical couplers400and403, however, unitary MMI optical couplers504and508have differential phase shifters located external to their respective waveguide structures510and520. In embodiments, the differential phase shifters are located or integrated on an external path (e.g.,535and557) optically coupled to the respective 2×2 unitary matrices. Unitary MMI optical couplers504and508each include a common phase shifter integrated within or on waveguide structures510and520. In embodiments, common phase shifters515and522are located in or integrated on substantially an entire optical path along respective waveguide structures510and520. In contrast, external phase shifters (517,519and525,527) are located on paths535and537that are external to optical paths525and565of respective waveguide structures510and520. Note that, in embodiments, due to having both common and differential phase shifters, unitary directional optical coupler100may be tuned with differential and common phase control modes.

FIGS. 6-8illustrate top and cross-sectional views of various embodiments of example 2×2 unitary directional optical couplers and 2×2 unitary MMI optical couplers. Note that in embodiments, the optical couplers are formed in crystalline silicon. Examples of waveguide materials include but are not limited to silicon, a thin silicon layer in SOI (silicon on insulator), glass, oxides, nitrides, e.g., silicon nitride, polymers, semiconductors or other suitable materials. In embodiments, waveguides in the optical couplers described in the FIGS. may be made of any medium that propagates a wavelength of light and surrounded with a cladding with a lower index of refraction. In some embodiments, waveguides may be formed on a buried oxide layer (BOX) layer of an SOI wafer with a top cladding layer over the waveguides. In embodiments, the top cladding layer includes silicon dioxide (SiO2) having an index of refraction of n=1.45, while a silicon-based waveguide has an index of refraction of, e.g., n=3.48. In embodiments, the optical couplers are formed via known lithography/etch methods associated with formation of optical waveguides on SOI wafers.

FIGS. 6A-6Fillustrate top and cross-sectional views of example 2×2 unitary directional optical couplers, in accordance with embodiments of the present disclosure.FIG. 6Aillustrates unitary directional optical coupler600which is the same or similar as unitary directional optical coupler100shown and described inFIG. 1(for brevity, description of some similar elements are not repeated). In embodiments, a dotted arrow199represents a plane through which a cross-section of unitary directional optical coupler600is shown inFIG. 6B. As shown, inFIG. 6B, first optical waveguide101and second optical waveguide103are single mode optical waveguide structures formed over a buried oxide layer (BOX)653on a silicon on insulator (SOI) wafer652. In the embodiment, a top cladding layer650is formed over first optical waveguide101and second optical waveguide103. In the embodiment, phase shifter107and phase shifter109are formed to abut or nearly abut respective first optical waveguide101and second optical waveguide103but do not cover first optical waveguide101and second optical waveguide103. In embodiments, an example width w of a gap108between waveguides101and103is 0.2-0.8 micrometers (μm). In the example ofFIG. 6A, first optical waveguide101and second optical waveguide103have heights of 0.2-0.4 μm (e.g., element679inFIG. 6B). Note that these widths and heights are only examples and any suitable heights or widths that are consistent with providing 2×2 unitary directional optical couplers with phase shifters to perform the unitary transformation are contemplated.

In some embodiments, after formation of phase shifters107and109, metal connections to control a tuning of the phase shifters using known methods are implemented. For example, various method include, but are not limited to, processes that include, e.g., resistive thin-film strip (doped silicon, SiN) or metal wire (TiW, Tungsten) as thermal phase shifters, or doped P+ regions and doped N+ regions to form p-i-n junctions as electro-optical phase shifters. For example,FIG. 6Eillustrates unitary directional optical coupler600after metal connections675and680are formed (note that similar or same elements have not been labeled for clarity in the FIGS), using known methods such as passivation layer (typical oxide layer, SiN) deposition, and pad openings for metal contacts and connections675and680. In various embodiments, metal connections675and680may include wire bonding, bump pads, or other suitable connections, coupled to allow a tenability of phase shifters107and109. In embodiments, electro-optic tuning of phase shifters107and109control application of weights being applied in matrix multiplication in the unitary transformation.

In an embodiment, shown inFIG. 6C, is another unitary directional optical coupler603. As shown, unitary directional optical coupler603includes a phase shifter617and phase shifter619that cover at least a top portion of first optical waveguide and a second optical waveguide605and607. In embodiments, a dotted arrow699represents a plane through which a cross-section of unitary directional optical coupler603is shown to the right of optical coupler603inFIG. 6D. As shown, phase shifters617and619are formed over a buried oxide layer (BOX)753over a silicon on insulator (SOI) wafer752. A top cladding layer750is shown above phase shifters617and619. As noted above, phase shifters617and619are formed to cover at least a portion of respective first optical waveguide605and second optical waveguide607.

After formation of phase shifters617and619, metal connections to control a tuning of the phase shifters are formed. For example,FIG. 6Fillustrates unitary directional optical coupler603after metal connections775and780are formed (note that similar or same elements have not been labeled for clarity in the FIGS). In various embodiments, metal connections775and780may include wire bonding, bump pads, or other suitable connections, to allow a tunability of phase shifters617and619.

In embodiments, phase shifter107and phase shifter109ofFIG. 6Aare PN-diode-based phase shifters or thermal based phase shifters. Note that in other embodiments, phase shifters617and619ofFIG. 6Cmay cover varying portions of first optical waveguide605and second optical waveguide607.

FIGS. 7A-7Cillustrate top and cross-sectional views of a 2×2 unitary MMI optical coupler, in accordance with embodiments of the present disclosure.FIGS. 7A-7Cillustrate embodiments associated with methods of forming phase shifters of a unitary MMI optical coupler.FIG. 7Aillustrates a unitary MMI optical coupler similar to as shown and described inFIG. 4(note that description of similar elements may not be repeated). In embodiments a dotted arrow799represents a plane through which a cross-section of unitary MMI optical coupler400is shown inFIG. 7B. As seen inFIG. 7B, unitary MMI optical coupler400is formed over a buried oxide layer (BOX)453on a silicon on insulator (SOI) wafer452. In embodiments, phase shifters407and409are formed to cover at least a portion of MMI waveguide structure410. In some embodiments, MMI waveguide structure410is a waveguide that is wide compared to, e.g., first optical waveguide401and second optical waveguide403, and includes a width Weof, for example, 2-10 μm and a height h of 0.2-0.4 μm. In the embodiment, additional phase shifter408is formed over (or integrated above) MMI waveguide structure410. After formation of the phase shifters, metal connections to control a tuning of the phase shifters are formed. For example,FIG. 7Cillustrates MMI optical coupler400after metal connections422are formed. In various embodiments, metal connections422may include wire bonding or bump pads coupled to tunable phase shifters of MMI optical coupler400. Although six metal connections are shown, only metal connection422is labeled for clarity in the FIGS.

Note that an electro-optical tuning applied through the metal connections allows the modes of the first optical signal and the second optical signal to interfere in the MM waveguide to output an optical signal at a power ratio that can be adjusted according to U(2) matrix algebra.

FIGS. 8A-8Cillustrate top views and cross-sectional views of another 2×2 unitary MMI optical coupler, in accordance with another embodiment of the present disclosure.FIGS. 8A-8Care associated with a method of forming phase shifters in a unitary MMI optical coupler.FIG. 8Ashows a top view of a unitary MMI optical coupler similar to that ofFIGS. 7A-7CandFIG. 4, with the exception that a first and a second phase shifter are formed next to MMI waveguide structure810(rather than covering a portion of MMI waveguide structure810). InFIG. 8A, a dotted arrow899represents a plane through which a cross-section of a unitary MMI optical coupler800is shown inFIG. 8B. As seen inFIG. 8B, unitary MMI optical coupler800is formed over a buried oxide layer (BOX)853on a silicon on insulator (SOI) wafer852. In embodiments, phase shifters807and809are formed next to MMI waveguide structure810. In the embodiment shown, a third, or additional, phase shifter808is formed over (or integrated above) MMI waveguide structure810.

After formation of the phase shifters, metal connections to control a tuning of the phase shifters807and809are formed. For example,FIG. 8Cillustrates unitary MMI optical coupler800after metal connections822are formed. In various embodiments, metal connections822may include wire bonding or bump pads coupled to tunable phase shifters807,808, and809of MMI optical coupler800. Although six metal connections are shown, only metal connection822is labeled for clarity in the FIGS.

Note that phase shifters407,409and807,808, and809ofFIGS. 7A and 8Amay include any suitable type of phase shifter such as, but not limited to, PN-junction diode phase shifters or thermal heater phase shifters. Furthermore, as noted previously, a number and configuration of phase shifters may vary. For example, in various embodiments, a plurality of phase shifters may be integrated on MMI waveguide structure410or810in a successive arrangement (not shown).

FIG. 9illustrates examples of a first matrix multiplier and a second matrix multiplier having a plurality of optical unitary matrices coupled together. In embodiments, the unitary optical matrices are coupled together to form matrix multipliers having a plurality of n optical inputs and a plurality of n optical outputs. In embodiments, the plurality of 2×2 unitary optical matrices are optically coupled to receive an array of optical signal inputs and to linearly transform the plurality of optical signal inputs into an array of optical signal outputs, wherein each of the plurality of 2×2 unitary optical matrices include a first optical waveguide and a second optical waveguide coupled to converge and diverge along an optical path.

In embodiments, matrix multiplier901is a larger unitary optical matrix that includes a plurality of 2×2 unitary directional optical matrices902(e.g., similar or the same as directional optical coupler100ofFIG. 1), while matrix multiplier903includes a plurality of 2×2 unitary multi-mode interference (MIMI) optical couplers904(e.g., similar or the same as the example 2×2 unitary (MMI) optical couplers ofFIG. 4). Note that for clarity in the FIG., only one of 2×2 directional optical matrices902(e.g., 2×2 directional optical coupler100ofFIG. 1) and one of 2×2 unitary MMI optical couplers904is labeled. For matrix multiplier901, a plurality of 2×2 directional optical matrices902are optically coupled together to receive an array of optical signal inputs at905inFIG. 8and to linearly transform the plurality of optical signal inputs into an array of optical signal outputs907. Similarly, for matrix multiplier903, a plurality of unitary MMI optical couplers904are coupled together to receive an array of optical signal inputs at911to linearly transform the plurality of optical signal inputs into an array of optical signal outputs913.

Note that in various embodiments, the matrix multipliers include any of, or any suitable combination of, different types of 2×2 optical matrices, such as the 2×2 unitary directional optical couplers and 2×2 unitary MMI optical couplers as described and shown in previousFIGS. 1-8. For example, in various embodiments, the matrix multipliers include a plurality of 2×2 unitary adiabatic directional optical couplers such as the 2×2 unitary adiabatic directional optical coupler ofFIG. 2, 2×2 unitary directional optical couplers and adiabatic directional optical couplers having one or more common or differential phase shifters ofFIG. 3, or 2×2 unitary multi-mode interference (MMI) optical couplers having one or more of differential phase shifters and/or common phase shifters ofFIG. 5.

Note that the array of optical signal inputs905for matrix multiplier901(and optical signal inputs911for matrix multiplier903) include n optical inputs and n optical signal outputs where n=8. In embodiments, the matrix multipliers each include n (n−1)/2 2×2 unitary optical matrices (e.g., n (n−1)/2 2×2 optical matrices). Although n=8 inFIG. 9for both matrix multiplier901and903, it should be understood that 8 is only an example and n is any number of optical inputs and optical outputs suitable for an application. In embodiments, n is 2, 4, 8, 16, 32, 64, 128, or 256. It is further understood that couplings as in matrix multiplier901and903have been simplified in order to conceptually illustrate optical connections between 2×2 directional optical matrices902or unitary multi-mode interference (MMI) optical couplers904. The matrix multiplier can have n optical inputs and m output outputs, n may be not equal to m where n, m=2, 3, 8, 16, 32, 64, 128 or 256, and the matrix multiplier includes n (m−1)/2 2×2 unitary optical matrices.

Accordingly, as described in connection withFIGS. 2-8, each of 2×2 directional optical matrices902and 2×2 unitary multi-mode interference (MMI) optical couplers904each include a first optical waveguide and a second optical waveguide coupled along an optical path. Furthermore, for the embodiments, a plurality of tunable optical phase shifters (e.g., as described in connection withFIGS. 1-8) are included along the optical path of each of the first optical waveguide and the second optical waveguide in each of the plurality of 2×2 unitary optical matrices to phase shift an optical beam to linearly transform the array of optical signal inputs into the array of optical signal outputs.

FIG. 10is a context diagram that shows a nonlinear optical device within a layer of an ONN included on a photonics integrated circuit (PIC) that will be discussed in connection withFIGS. 11-17, in accordance with various embodiments. In embodiments, the ONN includes one or more layers each including a plurality of optical unitary matrix multipliers followed by optical nonlinear optical devices implementing a nonlinearity function. Integrated photonic device100shows an ONN1002that includes one or more layers1004having multiple optical signal inputs1006and multiple optical signal outputs1008. In this example, each layer1004has 32 optical signal inputs1006and 32 optical signal outputs1008. In other embodiments the number of optical signal inputs1006or optical signal outputs1008may vary. In embodiments, the ONN1002may be provided as an integrated circuit on the integrated photonic device1000.

Within the ONN1002, a laser diode array (LDA)1010together with optical modulators1012(hereinafter referred to as “modulator1012”) provides optical input to a first layer1005. A photodetector array1014will receive optical output from the third layer1007, and convert that output into digital signals. In this example, light signals are sent from layer11005, to layer21004, and then to layer31007. Each layer is made up of an optical unitary matrix multiplier (that may include a plurality of optical unitary matrix multipliers) and non-linear optical devices (e.g., nonlinear optical amplifiers1024described below). In embodiments, the ONN1002including array (LDA)1010, modulator1012, multiple layers1005,1004,1007, and PDA1014can be implemented in a heterogeneously integrated photonics circuit, such as a single silicon photonics die or single semiconductor substrate1050.

Diagram1004ashows various components of the optical unitary matrix multiplier unit within layer21004, which includes three optical unitary matrix multipliers1018,1020,122that are composed of a plurality of optical unitary matrices (e.g., matrix multipliers including 2×2 unitary directional optical couplers and/or 2×2 unitary MMI optical couplers as described and shown in previousFIGS. 1-9). As shown, the light signals flow out of the Unoptical unitary matrix multiplier1022and into a plurality of nonlinear optical devices1024for each layer1004.

Nonlinear optical amplifiers1024may be needed to be coupled to the optical unitary matrix multiplier1022due to the linear nature of the optical signal processing from the optical unitary matrix multipliers1018,1020,1022. The optical signal, including noise added to the optical signal, may be linearly increased during operation of the ONN1002, and may result in a final signal intensity from the Unoptical unitary matrix multiplier1022that is too high. This signal intensity may cause optical inputs to overload a subsequent layer1007, or overload the PDA1014.

The nonlinear optical amplifier1024may comprise multiple nonlinear optical devices. An example nonlinear optical device1028is shown inFIG. 10to the right of the layer1004(blown-up area1027of the nonlinear amplifier1024). An optical input signal1025into the device1028may be transformed into an optical output signal1026of a particular nonlinear optical device1028shown with respect to area1027. The term “amplifier” is used in a broad sense here. The input signal1025may need to be amplified in a linear way, amplified in a non-linear way, as well as saturated and attenuated, and/or otherwise “cleaned up” in order for the resulting optical signal output1026to be more distinguishable. Other functions may include light rectifying and saturating for the resulting optical signal output for high classification and predication in the ONN layers. These functions are explained further in reference toFIG. 2.

he equation I out=f(Iin)eiΔϕon the output of1026shown inFIG. 10defines the overall optical signal input to optical signal output nonlinear activation function, where f is the optical intensity function of nonlinear optical device1028as a function of optical signal input power Iin; and Δϕ is the phase change from optical signal input to optical signal output generated by the non-linear optical device1028. The intensity function f includes optical amplifying, saturating, rectifying and attenuating, and/or a combination of these functions, or any types of similar function to serve as optical input to optical output nonlinear activation functions. In embodiments, a few criteria are to be met with respect to nonlinear optical device1028. First, the optical nonlinear activation may need active feedback control to emulate the arbitrary layers matrices and to classify and predict performance. Examples of active control are bias current, voltage and/or phase tuning operation for activation functions in optical amplifying, attenuating and saturating. Second, low electrical power consumption in each optical nonlinear device is typically determined by the biasing current times the biasing voltage applied on the nonlinear optical device1028, and it is desired to reach power efficiency in ONNs. Third, various optical nonlinear functions f can be implemented in the optical domain with associated IC driver and firmware algorithms, similar to various CMOS IC-based nonlinear functions.

For example, if the signal output1026level represents 8 bits, it may be desirable for the nonlinear optical device1028to clean up the representation of a low bit to 0, and a high bit to be put into the upper limits as a saturation function. This will enhance the performance of optical signal output to proceed to the next layer in the linear functions of the various optical matrix multipliers.

In embodiments, the nonlinear optical devices provide optical amplification to compensate for waveguide propagation loss needed to emulate the multiple layers of the ONN. In embodiments, a III-V gain medium is bonded to silicon photonics to provide amplification, where the gain medium has both linear and nonlinear amplification functions when input power reaches a saturation level. The amplification function may include a multi-quantum well medium to increase efficiency. In embodiments, a carrier-injection pin diode can be added to couple with the amplification function to provide light attenuation control to not overload the subsequent layer or photodiode array (PDA).

FIG. 11is a block diagram of an overview of an electronic circuitry1150and an integrated photonics device or photonics integrated circuit (PIC)1100. In embodiments, some or all functions of electronic circuitry1150are integrated together into a single EIC die (e.g.,FIGS. 13-16) or in different combinations as discrete EIC dies with PIC1100(e.g.,FIG. 12B) in an optical accelerator package. In embodiments, the single EIC die or discrete EIC dies will assist in supporting data loading and offloading from an optical matrix multiplier of PIC1100. In embodiments, the PIC is included on a single semiconductor substrate and includes at least an optical unitary matrix multiplier, an array of light sources, and an array of optical modulators and an array of photodetectors integrated in the single semiconductor substrate. In embodiments, the PIC, the array of light sources, e.g., hybrid lasers, and other optoelectronic components are integrated into the single semiconductor substrate.

Referring now to PIC1100, which includes an optical matrix multiplier1105and an array of light sources, such as, e.g., lasers1103in a semiconductor substrate, e.g., silicon substrate1101, to generate an array of light signals or optical signals. In embodiments, lasers1103includes any suitable light source such as, e.g., lasers or hybrid lasers (e.g., hybrid bonded lasers on a silicon photonics chip including silicon substrate1101) such as indium phosphide (InP) lasers. PIC1100further includes an array or plurality of optical modulators1110coupled to lasers1103to receive the array of optical signals. The optical modulator1110converts the electrical data into modulated optical signals to generate an array of optical signal inputs. In various embodiments, optical modulators may be Mach-Zehnder interferometers, optical ring modulators, or other suitable high-speed optical modulators. In embodiments, after modulation, optical modulators1110provide a plurality of optical signal inputs to optical matrix multiplier1105integrated in silicon substrate1101. As shown in connection with, e.g.,FIG. 9, optical unitary matrix multiplier1105includes a plurality of 2×2 unitary optical matrices optically interconnected. In the embodiment, optical unitary matrix multiplier1105performs matrix multiplication to linearly transform optical signal inputs into an array or plurality of optical signal outputs.

As shown inFIG. 11, PIC1100also includes an array or plurality of photodetectors1107(“photodetectors1107”) such as, e.g., waveguide photodetectors, avalanche photodetectors, coupled to detect the optical signal outputs. Non-linear optical devices1106(e.g., similar to as shown and described in connection withFIG. 10) are coupled to amplify or attenuate optical signal outputs prior to being detected by photodetectors1107, which convert the optical signal outputs to photocurrent. In embodiments, electronic amplifiers such as transimpedance amplifiers (TIA) (not shown) are coupled to photodetectors1107to receive photocurrent from photodetectors1107to further amplify the electrical signal outputs.

Electronic circuitry1150is coupled to PIC1100via radiofrequency (RF) and direct current (DC) routing interconnections1168(e.g., an interconnect bridge or other multi-die interconnection structure). Electronic circuitry1150includes weights1161, a data pipeline1163, control logic1165, post-processing unit1169, a memory (e.g., SRAM)1167, and a high-speed interface1175. In embodiments, memory access and management units1171and a controller1173for CPU control are included in electronic circuitry1150. In some embodiments, memory access and management units1171includes e.g., a Direct Memory Access unit (DMA) and/or a memory management unit (MMU). In embodiments, memory access and management units1171transfer data to and from memory1153and/or data pipeline1163(e.g., activation buffers and the data included in data pipeline1163) as needed.

CPU1155is coupled to electronic circuitry1150via a high speed input/output (I/O) bus1160(e.g., the latest generation of Peripheral Component Interconnect Express (PCIe) or other high-speed bus). In embodiments, memory1153(e.g., Double Data Rate Synchronous Dynamic Random-Access Memory (DDR SDRAM)) provides weights (e.g., initial training weights, staged weights, reuse weights) as well as instructions to be implemented by control logic1165. In embodiments, memory1153is coupled to provide a digital-to-analog converter circuitry (DAC)1125with weights for optical matrix multiplier1105via a relatively low speed link. In embodiments, data incoming from CPU1155(e.g., data values associated with applications, e.g., speech recognition, computer vision, multimedia, and the any suitable machine learning application) to be analyzed by an inference or predictive model of an ONN is provided by CPU1155via I/O bus1160to join data pipeline1163. In embodiments, data pipeline1163provides a DAC1117with real-time data or data input via interface1175that is to be input to optical matrix multiplier1105via optical modulators1110.

In embodiments, for an N×M matrix of optical matrix multiplier1105, optical modulators1110encode an N-dimensional input vector (“vector”) of values, x1, x2, . . . xN, into the array of optical signal inputs. Optical matrix multiplier1105then applies the weights input by DAC1125to perform matrix multiplication, resulting in a transformation on the optical signals. Optical matrix multiplier1105then provides optical output signals to non-linear optical devices (amplifiers and/or attenuators)1106for non-linear transformation. In embodiments, photodetectors1107then detect the optical signal outputs and convert the optical signal outputs to photocurrent, which is amplified by transimpedance amplifiers (TIA), and then sent to analog-to-digital converter (ADC) circuitry1118.

Accordingly, in embodiments, the ADC circuitry (indicated at ADC1118) converts the optical signal outputs (“outputs”) to electrical signals as real time data that are returned via a high speed link to electronic circuitry1150. In embodiments, the outputs may be provided to data pipeline1163, undergo post-processing at post process1169, and returned to CPU1155or SRAM1167for next steps. In embodiments, e.g., during a training model in a learning stage of an ONN application, where weights are being updated, the cycle may be repeated until weighted output errors associated with a set of data (such as training data) are sufficiently reduced.

FIG. 12Aillustrates a top view of the functional block diagram of electronic circuitry1150ofFIG. 11and a photonics integrated circuit (PIC)1200. PIC1200is similar or the same as PIC1100ofFIG. 11, however, in PIC1200is coupled with a plurality of discrete electronic integrated circuit (EIC) dies that integrate one or more elements of electronic circuitry1150. In the embodiment ofFIG. 12B, PIC1200is a single flip-chip PIC stacked or proximally located to the plurality of discrete EIC dies. In embodiments, the plurality of discrete EIC dies are stacked proximally to PIC1200to provide pre- and post-processing of optical signal inputs and optical signal outputs including optical to electrical and electrical to optical transduction. Note that PIC1200and PIC1100ofFIG. 11include similar elements, and the descriptions of certain elements are repeated only as necessary below.

In the embodiment ofFIG. 12B, a PIC die-stack assembly1250is located on top of a discrete EIC die-stack assembly1245of a single optical accelerator package. In the embodiment, PIC1200is disposed on a redistribution layer1241(redistribution layer1241includes, e.g., a silicon substrate, silicon interposer, or the like). InFIG. 12B, the plurality of discrete EIC dies include a DAC/ADC die1227, a laser driver and optical modulator driver die1237, and a controller die1239. In embodiments, DAC/ADC die1227also includes weights (e.g., weights that can be sent via a weight buffer that reads/writes to/from an SRAM on the EIC or PIC which connects to a DDR memory interface) to be provided to optical matrix multiplier1205. As shown, in the present configuration, controller die1239is disposed on a package substrate, e.g., optical accelerator package substrate1235(“substrate1235”) and below a coupling structure, e.g., a silicon interposer1233. As seen inFIG. 12B, silicon interposer1233includes vias1232which connect to connection pads1242or bumps1230. Note that only one connection pad, connector, bump or via, may be labeled in the FIG. for clarity. In embodiments, the connection pads, connectors, bumps, or vias assist in providing connections equivalent to RF/DC routing connections1168ofFIGS. 11 and 12A.

Integration of the discrete EICs into the optical accelerator package as described may provide higher bandwidth, higher density and lower power consumption due to a proximal location of radiofrequency (RF) interfaces of the PIC and EIC. Note that in embodiments, as an example, input data from CPU1155follows a path1212to DAC/ADC die1227. In embodiments, after DAC/ADC die1227converts the input data from digital to analog format, it is received by laser driver and optical modulator driver die1237to be modulated into optical signal inputs for optical matrix multiplier1205. In the embodiment ofFIG. 12A, controller die1239receives instructions from CPU1155to be implemented by PIC1200as well as performs, e.g., performance management integrated circuit (PMIC) functions. In some embodiments, controller die1239controls drivers included in e.g. laser driver and optical modulator driver die1237. In other embodiments, controls for the lasers and the optical modulators are located in DAC/ADC die1227. In embodiments, an output path (not shown) of output data from PIC1100between and the plurality of discrete EIC dies may follow a path moving downwards from PIC1200and form a closed loop within the optical accelerator package.

Note that the configuration of the plurality of discrete EIC dies in relation to PIC1200shown inFIG. 12Bare merely examples. In other embodiments, for example, DAC/ADC die1227, laser driver and optical modulator driver die1237, and controller die1239, are disposed at any relative location to PIC1200that facilitate pre- and post-processing of optical signal inputs and optical signal outputs and including optical to electrical and electrical to optical transduction Furthermore, the functions of each of the plurality of discrete EIC dies may be combined with one or more other functions of the other discrete dies or those not discussed herein but that assist in the pre-and post-processing of the optical signal inputs and optical signal outputs.

Referring now toFIG. 13which is a side view of an embodiment of a PIC1300(similar to PIC100ofFIG. 11) vertically stacked over a single integrated EIC. In embodiments, a single integrated EIC1318integrates some or substantially all elements of electronic circuitry, e.g., electronic circuitry1150ofFIG. 11. In the embodiment ofFIG. 13, PIC1300is a single flip-chip PIC (of a PIC die-stack assembly1350) on top of single integrated EIC1318(of a single EIC die-stack assembly1340) which integrates some or substantially all elements of electronic circuitry1150. Note that PIC1300includes the same or similar elements as PIC1100, e.g. an array of light sources or lasers1303coupled to plurality of optical modulators1310(“optical modulators1310”), which provide modulated optical signal inputs to an optical matrix multiplier1305integrated in silicon substrate1301. PIC1300also includes non-linear (NL) optical devices1306and photodetectors1307. In the embodiment, PIC1300is disposed on a redistribution layer1341(redistribution layer1341includes, e.g., a silicon substrate, silicon interposer, or the like) which is connected via connectors1338to pads1342and vias1332of silicon interposer1333. Note that only one connector, pad, and via, may be labeled in the FIG. for clarity. As shown, single integrated EIC1318is disposed between package substrate1335and silicon interposer1333.

As shown, in embodiments, input data from a CPU, e.g., CPU1155ofFIG. 11, follows a path1312to single integrated EIC die1318and ultimately to PIC1300. In the embodiment single integrated EIC die1318includes e.g., some or substantially all functions of electronic circuitry1150ofFIG. 11. In embodiments, pre- and post-processing of optical signal inputs and optical signal outputs includes at least, electro-optical and opto-electrical conversion of data provided to and received from the PIC. In various embodiments, single integrated EIC die1318includes DAC circuitry (e.g., to perform functions similar to as described in connection with DAC1117and1125ofFIG. 11) and ADC circuitry (e.g., to perform functions similar to as described in connection with ADC1118ofFIG. 11), laser and optical modulator drivers (e.g., to perform functions similar to as described in connection with ADC1118ofFIG. 11), control for laser and optical modulator drivers (to perform functions similar to as described in connection with (PMIC) controller1102or ADC1118and DAC1117/1125ofFIG. 11), CPU control circuitry, memory (e.g., SRAM1167ofFIG. 11) for storage of weights and CPU instruction.

Referring now toFIG. 14which is a side view of an embodiment of a PIC (similar to PIC1100ofFIG. 11) and a CPU on a similar substrate vertically stacked over a single integrated EIC. The configuration is similar to that ofFIG. 13, with the exception of the CPU including an SRAM memory (to perform functions similar to CPU1155ofFIG. 11) included on a common substrate with PIC1400. In embodiments a single integrated EIC1418integrates some or substantially all elements of electronic circuitry, e.g., electronic circuitry1150ofFIG. 11. In the embodiment ofFIG. 14, PIC1400is a single flip-chip PIC (see PIC die-stack assembly1450) vertically stacked above single integrated EIC1418(of single EIC die-stack assembly1450) which integrates some or substantially all elements of electronic circuitry1150ofFIG. 11. As shown in the FIG., a CPU and SRAM combination1456is located on a common substrate1435with PIC1400. In embodiments, the CPU is a flip-chip CPU that performs functions similar to CPU1155ofFIG. 11. In some embodiments, SRAM and/or CPU and SRAM combination includes memory in a chiplet or high-efficiency (angular division multiplexing (ADM) memory chiplets or RAMBO™ memory chiplet format.

Note that PIC1400includes the same or similar elements as PIC1100ofFIG. 11, e.g. array of light sources or lasers1403coupled to plurality of optical modulators1410(“optical modulators1410”), which provide optical signal inputs to optical matrix multiplier1405integrated in silicon substrate1401. PIC1400also includes non-linear (NL) optical devices1406and photodetectors1407. In the embodiment, PIC1400is disposed on a redistribution layer1441(redistribution layer1441includes, e.g., a silicon substrate, silicon interposer, or the like) which is connected via connectors1438to pads1442and vias1432of silicon interposer1433. Note that only one connector, pad, and via, may be labeled in the FIG. for clarity. As shown, single integrated EIC1418is disposed on package substrate1435.

Note that in embodiments, input data from CPU and SRAM combination1456follows a path1412downward through a silicon interposer1433to single integrated EIC die1418and then upwards to PIC1400. In the embodiment, single integrated EIC die1418includes e.g., some or substantially all functions of electronic support circuitry1150ofFIG. 11. In embodiments, single integrated EIC die1418includes DAC circuitry (e.g., to perform functions similar to as described in connection with DAC1117and1125ofFIG. 11), ADC circuitry1118(e.g., to perform functions similar to as described in connection with ADC1118ofFIG. 11), laser and optical modulator drivers and control of (e.g., to perform functions similar to as described in connection with controller1102ofFIG. 11), control for laser and optical modulator drivers (also to perform functions similar to as described in connection with (PMIC) controller1102ofFIG. 11), CPU control circuitry, memory (e.g., SRAM1167ofFIG. 11) for weights and other functions not performed by CPU and SRAM combination1456.

Referring now toFIG. 15which is a side view of another embodiment of a PIC and a CPU vertically stacked over a single integrated EIC. The configuration is similar to that ofFIG. 14, where a CPU (e.g. CPU and SRAM combination) is included on a same substrate as PIC1500. In contrast, however, toFIG. 14, single integrated EIC1518, rather than sitting above a package substrate, is integrated into a package substrate1535. Similar toFIGS. 13 and 14, in embodiments, single integrated EIC1518integrates some or substantially all elements of electronic circuitry, e.g., electronic circuitry1150ofFIG. 11. In the embodiment ofFIG. 15, PIC1500is a single flip-chip PIC (see PIC die-stack assembly1550) on a redistribution layer1541that is disposed on package substrate1535that integrates single integrated EIC1518of single EIC die-stack assembly1550. As shown in the FIG., a CPU and SRAM combination1556is located on a common substrate1535with PIC1400. In embodiments, the CPU is a CPU with functions similar to CPU1155ofFIG. 11.

Note that in some embodiments, input data from CPU and SRAM combination1556follow a path1512to single integrated EIC die1518and then to PIC1500. In the embodiment single integrated EIC die1518includes e.g., some or substantially all functions of electronic circuitry1150ofFIG. 11. In embodiments, single integrated EIC die1518includes DAC circuitry (e.g., to perform functions similar to as described in connection with DAC1117and1125ofFIG. 11), ADC circuitry (e.g., to perform functions similar to as described in connection with ADC1118ofFIG. 11), laser and optical modulator drivers (e.g., to perform functions similar to as described in connection with controller1102or ADC1118and DAC1117,1125ofFIG. 11), control for laser and optical modulator drivers (also to perform functions similar to as described in connection with controller1102or ADC1118and DAC1117,1125ofFIG. 11), CPU control circuitry, and memory (e.g., SRAM1167ofFIG. 11for weights and other functions not performed by CPU and SRAM combination1556).

Referring now toFIG. 16which is a side view of an embodiment including a single integrated EIC1618stacked over a PIC1600(similar to PIC1100ofFIG. 11). Similar to previousFIGS. 13-15, in embodiments, single integrated EIC1618integrates some or substantially all elements of electronic support circuitry, e.g., electronic support circuitry1150ofFIG. 11. In the embodiment ofFIG. 16, PIC1600is a single flip-chip PIC of a PIC die-stack assembly1650. In the embodiment, PIC1600is disposed on a redistribution layer1641disposed toward a bottom of an IC optical accelerator package on substrate1635.

Note that PIC1600includes the same or similar elements as PIC1100, e.g. an array of light sources or lasers1603coupled to plurality of optical modulators1610(“optical modulators1610”), which provide optical signal inputs to an optical matrix multiplier1605integrated in silicon substrate1601. PIC1600also includes non-linear (NL) optical devices and photodetectors1607. In the embodiment, single integrated EIC1618is connected via connectors1638A to pads1642and vias1632of silicon interposer1633. Note that only one connector, pad, and via, may be labeled in the FIG. for clarity. As shown, single integrated EIC1618is located at a top of the configuration and thus may allow easier access to pins of EIC1618as well as thermal advantages for EIC1618.

Note that in embodiments, input data from a CPU (e.g. CPU1155ofFIG. 11) follows a path1612upwards through silicon interposer1633to single integrated EIC die1618and then downwards to PIC1600. In the embodiment single integrated EIC die1618includes e.g., some or substantially all functions of electronic support circuitry1150ofFIG. 11. For example, in embodiments, single integrated EIC die1618includes DAC circuitry (e.g., to perform functions similar to as described in connection with DAC1117and1125ofFIG. 11), ADC circuitry (e.g., to perform functions similar to as described in connection with ADC1118ofFIG. 11), laser and optical modulator drivers (e.g., to perform functions similar to as described in connection with controller1102or ADC1118and DAC1117,1125ofFIG. 11), control for laser and optical modulator drivers (to perform functions similar to as described in connection with (PMIC) controller1102or ADC1118and DAC1117,1125ofFIG. 11), CPU control circuitry, and memory (e.g., SRAM1167ofFIG. 11to store weights, etc.).

Note that the configuration of the plurality of single integrated EIC die in relation to the PICS and/or a CPU/SRAM combination shown inFIGS. 13-16are merely examples. In various embodiments, any suitable combination or configuration (e.g., vertically stacked or side-by-side) of the single integrated EIC die, PIC, and/or a CPU/SRAM that facilitates pre- and post-processing of optical signal inputs and optical signal outputs are contemplated. Furthermore, the included functions of the single integrated EIC dies may include other functions not discussed herein but that also assist in the pre-and post-processing of the optical signal inputs and optical signal outputs. In addition, the input data paths are merely examples and it is understood that paths for input data and output data from and between the single integrated EIC die and the PIC will vary. In embodiments, the paths provide radiofrequency (RF) and DC interfaces between the PIC, EIC and/or CPU/SRAM within a single optical accelerator package that offers higher efficiency and speed relative to separate configurations. In embodiments, a relatively smaller size of the PIC that includes the matrix multipliers ofFIGS. 1-9allows integration into a single package with the EIC configurations shown herein. Integration of the EIC into the optical accelerator package as described inFIGS. 12-16as described above may provide higher bandwidth, higher density and lower power consumption due to a proximal location of radiofrequency (RF) interfaces of the PIC and EIC.

FIG. 17illustrates an example computing device1701suitable for use with an integrated photonics device or PIC1700(e.g., similar to or the same as PICS1100-1600of respectiveFIGS. 11-16), in accordance with various embodiments as described herein. In embodiments, PIC1700includes an optical neural network (ONN) integrated circuit (IC) including an array of light sources and an optical matrix multiplier in a semiconductor substrate. In embodiments, the array of light sources generates an array of light signals and PIC1700further includes an integrated plurality of optical modulators to receive the array of light signals and modulate data onto the array of light signals and provide optical signal inputs to the optical matrix multiplier. In embodiments, the optical matrix multiplier linearly transforms the plurality of optical signal inputs into an array of optical signal outputs. In embodiments, a processor coupled to the PIC provides the PIC with the data to modulate onto the array of optical signal inputs to be transformed by the optical matrix multiplier.

For example, as shown, computing device1701may include a one or more processors or processor cores1703and memory1704. In embodiments, memory1704may be system memory. For the purpose of this application, including the claims, the terms “processor” and “processor cores” may be considered synonymous, unless the context clearly requires otherwise. The processor1703may include any type of processors, such as a central processing unit CPU, a microprocessor, and the like. The processor1703may be implemented as an integrated circuit having multi-cores, e.g., a multi-core microprocessor. The computing device1701may include mass storage devices1706(such as diskette, hard drive, volatile memory (e.g., dynamic random-access memory (DRAM), compact disc read-only memory (CD-ROM), digital versatile disk (DVD), and so forth). In general, memory1704and/or mass storage devices1706may be temporal and/or persistent storage of any type, including, but not limited to, volatile and non-volatile memory, optical, magnetic, and/or solid state mass storage, and so forth. Volatile memory may include, but is not limited to, static and/or dynamic random-access memory. Non-volatile memory may include, but is not limited to, electrically erasable programmable read-only memory, phase change memory, resistive memory, and so forth. In embodiments, processor1703is a high performance or server CPU (e.g., CPU1155). In some embodiments, optical accelerator1788includes an IC optical accelerator package that also includes processor1703or CPU1155(e.g.,FIGS. 14 and 15).

The computing device1701may further include input/output (I/O) devices1708(such as a display (e.g., a touchscreen display), keyboard, cursor control, remote control, gaming controller, image capture device, and so forth) and communication interfaces1710(such as network interface cards, modems, infrared receivers, radio receivers (e.g., Bluetooth), and so forth). In some embodiments, the communication interfaces1710may include or otherwise be coupled with integrated photonics device1701, as described above, in accordance with various embodiments.

The communication interfaces1710may include communication chips that may be configured to operate the device1700in accordance with a Global System for Mobile Communication (GSM), General Packet Radio Service (GPRS), Universal Mobile Telecommunications System (UMTS), High Speed Packet Access (HSPA), Evolved HSPA (E-HSPA), or Long-Term Evolution (LTE) network. The communication chips may also be configured to operate in accordance with Enhanced Data for GSM Evolution (EDGE), GSM EDGE Radio Access Network (GERAN), Universal Terrestrial Radio Access Network (UTRAN), or Evolved UTRAN (E-UTRAN). The communication chips may be configured to operate in accordance with Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Digital Enhanced Cordless Telecommunications (DECT), Evolution-Data Optimized (EV-DO), derivatives thereof, as well as any other wireless protocols that are designated as 3G, 4G, 5G, and beyond. The communication interfaces1710may operate in accordance with other wireless protocols in other embodiments.

The above-described computing device1701elements may be coupled to each other via system bus1712, which may represent one or more buses. In the case of multiple buses, they may be bridged by one or more bus bridges (not shown). Each of these elements may perform its conventional functions known in the art. In particular, memory1704and mass storage devices1706may be employed to store a working copy and a permanent copy of the programming instructions for the operation of PIC1700and integrated or discrete EICs1780. The various elements may be implemented by assembler instructions supported by processor(s)1703or high-level languages that may be compiled into such instructions.

The permanent copy of the programming instructions may be placed into mass storage devices1706in the factory, or in the field, through, for example, a distribution medium (not shown), such as a compact disc (CD), or through communication interface1710(from a distribution server (not shown)). That is, one or more distribution media having an implementation of the agent program may be employed to distribute the agent and to program various computing devices.

The number, capability, and/or capacity of the elements1708,1710,1712may vary, depending on whether computing device1701is used as a stationary computing device, such as a server computer in a data center, or a mobile computing device, such as a tablet computing device, laptop computer, game console, or smartphone. Their constitutions are otherwise known, and accordingly will not be further described.

For one embodiment, at least one of processors1703may be packaged together with computational logic1722configured to practice aspects of optical signal transmission and receipt described herein to form a System in Package (SiP) or a System on Chip (SoC).

In various implementations, the computing device1701may comprise one or more components of a data center, a laptop, a netbook, a notebook, an ultrabook, a smartphone, a tablet, a personal digital assistant (PDA), an ultra mobile PC, a mobile phone, or a digital camera. In further implementations, the computing device1701may be any other electronic device that processes data.

According to various embodiments, the present disclosure describes a number of examples.

Example 1 includes an optical accelerator package, comprising a photonics integrated circuit (PIC), wherein the PIC includes an optical matrix multiplier to transform an array of optical signal inputs into an array of optical signal outputs; and an electronics integrated circuit (EIC) coupled to the PIC, wherein the EIC is heterogeneously integrated into the optical accelerator package in a manner to proximally provide pre- and post-processing of the optical signal inputs and the optical signal outputs provided to and received from the optical matrix multiplier of the PIC.

Example 2 includes the optical accelerator package of Example 1, wherein the EIC is stacked vertically above or below the PIC and the PIC includes the optical matrix multiplier and an array of light sources and an array of optical modulators integrated in the single semiconductor substrate.

Example 3 includes the optical accelerator package of Example 2, wherein the optical matrix multiplier comprises a plurality of 2×2 unitary optical matrices optically interconnected, wherein each 2×2 unitary optical matrix comprises a plurality of phase shifters to phase shift, split, or combine one or more of the optical signal inputs.

Example 4 includes the optical accelerator package of Example 1, wherein the optical signal inputs and the optical signal outputs provided to and received from the optical matrix multiplier unit include data provided to and received from a server central processing unit (CPU) coupled to the optical accelerator package.

Example 5 includes the optical accelerator package of Example 1, wherein the pre- and post-processing of the optical signal inputs and the optical signal outputs includes electro-optical and opto-electrical conversion of data provided to and received from the PIC.

Example 6 includes the optical accelerator package of Example 1, wherein the EIC further comprises drivers for a plurality of lasers and optical modulators included in the PIC and a controller to control the drivers.

Example 7 includes the optical accelerator package of Example 1, wherein the EIC further includes an SRAM memory to store a plurality of weights to be provided to the optical unitary matrix multiplier unit.

Example 8 includes the optical accelerator package of Example 1, wherein the EIC further includes control circuitry to implement control from the server central processing unit (CPU).

Example 10 includes the optical accelerator package of Example 1, wherein the EIC includes at least one of an analog to digital converter (ADC), digital to analog converter (DAC), laser driver, optical modulator driver, transimpedance amplifier (TIA), performance management integrated circuit (PMIC), central processing unit (CPU) controller circuitry, storage or pipeline for weights, and SRAM memory.

Example 11 includes an optical accelerator package, comprising: a photonics integrated circuit (PIC) die, wherein the PIC includes an optical matrix multiplier to transform an array of optical signal inputs into an array of optical signal outputs; and a plurality of discrete electronics integrated circuit (EIC) dies coupled to the PIC, wherein the plurality of EIC dies are stacked in a manner vertically above or below the PIC to proximally provide pre- and post-processing of the optical signal inputs and the optical signal outputs provided to and received from the optical matrix multiplier of the PIC.

Example 12 includes the optical accelerator package of Example 11, wherein the PIC is included on a single semiconductor substrate and includes the optical matrix multiplier and an array of light sources and an array of optical modulators and an array of photodetectors integrated in the single semiconductor substrate, wherein the PIC is optically self-contained without a need to connect optically with other photonics dies or optical assemblies

Example 13 includes the optical accelerator package of Example 11, wherein the plurality of discrete electronics integrated circuit (EIC) dies comprise discrete dies to provide one or more of analog-to-digital converter (ADC) functions, digital-to-analog converter (DAC) functions, TIA, performance management integrated circuit (PMIC), driver and driver control functions for optical modulators lasers, memory, and CPU control circuitry functions.

Example 14 includes the optical accelerator package of Example 11, wherein the optical signal inputs and the optical signal outputs provided to and received from the optical matrix multiplier unit include data provided to and received from a server central processing unit (CPU) coupled to the optical accelerator package.

Example 15 includes the optical accelerator package of Example 12, wherein the PIC and the EIC comprise a co-processor for the server CPU.

Example 16 includes a system for implementing an optical neural network (ONN), comprising: an optical accelerator package, including a photonics integrated circuit (PIC) die, wherein the PIC die includes an optical matrix multiplier to transform an array of optical signal inputs into an array of optical signal outputs; and an electronics integrated circuit (EIC) die coupled to the PIC die, wherein the EIC die is stacked in a manner vertically above or below the PIC die to proximally provide pre- and post-processing of the optical signal inputs and the optical signal outputs provided to and received from the optical matrix multiplier of the PIC die; and a central processing unit (CPU) coupled the optical accelerator package to provide the data to and from the optical accelerator package to be converted into the optical signal inputs and the optical signal outputs.

Example 17 includes the system of Example 16, wherein the EIC includes at least two of an analog to digital converter (ADC), digital to analog converter (DAC), laser drivers, optical modulator drivers, controller circuitry, and SRAM memory.

Example 18 includes the system of Example 16, wherein the EIC die provides substantially all functions required for pre-and post-processing of analog data provided between the PIC die and the CPU including optical-to-electrical and electrical-to-optical transduction.

Example 19 includes the system of Example 16, wherein the PIC includes the optical matrix multiplier, an array of light sources, and an array of optical modulators integrated in the single semiconductor substrate.

Example 20 includes the optical accelerator package of any one of Examples 16-19, wherein the optical matrix multiplier comprises a plurality of 2×2 unitary optical matrices optically interconnected, wherein each 2×2 unitary optical matrix comprises a plurality of phase shifters to phase shift, split, or combine one or more of the optical signal inputs.