Self-configuring high-speed multi-plane light conversion

Domain: physics
Published: 2025-12-08
Editorial outcome: minor_revision

Abstract
Multi-plane light converters (MPLCs)–also known as diffractive neural networks–convert an orthogonal set of optical fields into any other orthogonal set via a unitary transformation. MPLC design typically involves optimising a digital model. However, inherently high levels of complexity mean that even a minor mismatch between this model and the physically realised MPLC leads to a severe reduction in performance. Here we create a self-configuring MPLC, converging in minutes while automatically absorbing unknown misalignments and aberrations into the design. To achieve this we introduce ‘multi-plane wavefront shaping’–allowing multiple spatial light modes to be reshaped simultaneously. Convergence is accelerated via a high-speed MPLC platform incorporating a kHz-rate phase-only light modulator. Using this approach we demonstrate arbitrary optical transformations and universal mode sorters. Our work paves the way towards ultra-high-fidelity MPLCs with potential applications to optical communications, photonic computing and imaging.

Introduction
Spatial light modulators (SLMs) are the workhorses of high-dimensional light manipulation1. They are capable of arbitrarily patterning a beam of light across millions of independently tuneable pixels2. However, despite their high resolution, a single reflection from a planar two-dimensional (2D) SLM can only efficiently transform a single spatial light mode at a time. Yet the next generation of photonic technologies calls for the ability to efficiently modulate an entire basis of spatial light modes simultaneously: deterministically mapping a group of input spatial modes to a new group of output modes. Optical devices that can passively perform such spatial basis transformations have a diverse range of applications. Examples include spatial mode multiplexers for optical communication links3, multicasting reconfigurable optical switches4, mode sorters for far-field super-resolution imaging5,6, light unscramblers for visualising scenes hidden behind opaque media7,8, and matrix operators in emerging forms of classical and quantum optical computation architectures9,10.

So why is it not generally possible for spatial basis transformations to be achieved by a single reflection from an SLM? The root of the problem is that a different hologram is typically required to reshape each different mode incident onto an SLM. While these different holograms can be multiplexed and displayed on an SLM together11, each mode is diffracted from all multiplexed holograms, resulting in only a fraction of the light being transformed as desired12. This limitation affects all 2D planar light manipulation technologies regardless of their resolution, including liquid crystal SLMs, digital micro-mirror devices, deformable mirrors and metasurfaces. To overcome this issue, inherently three-dimensional (3D) light modulation architectures are called for13. At present, such technologies are still in their infancy. Photonic integrated circuits (PICs), composed of waveguide arrays with embedded phase shifters on chip, offer a way forward14–18. However, PICs are not yet widely available, and difficult to scale up to high dimensions. An emerging alternative technology is free-space multi-plane light conversion, which is the focus of this work.

Multi-plane light converters (MPLCs)19–23—which have more recently become known as linear diffractive neural networks24,25—consist of a cascade of planar diffractive elements (the ‘planes’, which here we also refer to as ‘phase masks’) separated by regions of free-space. Each phase mask imparts a carefully designed spatially-varying phase delay to light flowing through the device, and the diffraction in between each pair of phase masks allows energy to be exchanged laterally. In this way, input optical fields are sequentially processed and transformed into target output fields—emulating a fully 3D light processing architecture by coarse-graining it into a series of layers. Crucially, MPLCs can efficiently apply distinct transformations to multiple input modes simultaneously, thus achieving the spatial basis transformations that are much sought after in photonics26.

The design of an MPLC is a non-linear problem—the choice of phase profile on one plane being non-linearly dependent upon the phase profiles on planes further up- or downstream. Therefore, all phase masks must be jointly optimised, which is typically achieved via the process of inverse design27,28. A numerical model of the MPLC is iteratively optimised using adjoint methods that, in each iteration, efficiently determine how the phase of all pixels should be adjusted to improve the design23,29–31. This process is repeated until the design converges. Once designed, reconfigurable MPLCs can be implemented using multiple reflections from liquid crystal SLMs25,32–34.

However, as MPLCs are based on cascading planes, they are extremely sensitive to fabrication errors, which accumulate as light propagates through the device. This means that even a minor mismatch between the digital model used in the design phase and the physically realised optical system leads to a severe drop off in real-world MPLC performance30,35 (as demonstrated in Supplementary Section S8). Implementing an MPLC necessitates pixel-perfect alignment between the phase masks and the propagating fields on every plane, and simultaneous optimisation often of alignment degrees of freedom30. For optimal performance, a number of factors must be accounted for, including distortion of the input fields, phase aberrations of the planes themselves, and the imperfect response of the SLM (for example, problems arising from surface flatness, lack of parallelism between the optical surfaces within the SLM display, and crosstalk between neighbouring pixels36). These issues are exacerbated as the number of planes, and the complexity of their design, increases—holding back the scale of MPLCs and diffractive neural networks that have been successfully demonstrated to date.

To overcome these challenges, it is highly desirable to develop methods to optimise MPLCs and diffractive neural networks in-situ, circumventing the need to precisely match a digital model with the real physical system. Furthermore, granting self-aligning capabilities to free-space MPLCs would be beneficial for their real-world deployment, enabling high-fidelity operation to be maintained through varying environmental conditions (e.g., temperature changes) that would otherwise risk misaligning these complex optical systems. Advances in this area also push forward the development of physical neural networks that can be trained in-situ37, and adaptive optical technology capable of reversing the mixing of signals transmitted through complex scattering media—an emerging concept with many future imaging and communications applications7,8.

In this work we demonstrate a self-configuring free-space MPLC. Despite the large number of parameters to be optimised (up to 32,400 in our experiments), our proof-of-principle device converges on a timescale of minutes using a method in which light is only transmitted in one direction through the optical system. To make this possible, we develop a bespoke optimisation algorithm, and introduce a fast-switching MPLC platform based on a recently developed microelectromechanical system (MEMS)-based SLM38, shown schematically in Fig. 1a—allowing millions of MPLC configurations to be rapidly explored. This optimisation scheme naturally accounts for the physical characteristics of all system components by absorbing any unknown misalignments and aberrations into the final design. Our work paves the way towards a new generation of high-dimensional and ultra-high-fidelity fast-switching MPLCs.

Results
In-situ MPLC optimisation algorithm
We first describe how to automatically configure an MPLC to transform a single input spatial mode to a target output mode (an example is shown in Fig. 1b). Our approach is inspired by methods developed to control the propagation of light through complex scattering media—using a concept known as wavefront shaping39. Wavefront shaping can be accomplished by first measuring the transmission matrix (TM) of the medium40—a linear matrix operator describing how an arbitrarily shaped field incident on one side of a complex medium will have been reshaped by the time it emerges from the other side. The TM represents a digital model of the medium’s optical response, and once known, this model can be used to calculate how the input field should be shaped to generate a target field at the output41.

In our case, we treat the MPLC itself as the complex medium. We measure the TM from a particular MPLC plane to the output, and calculate how the phase profile of the plane in question should be updated to generate the desired output field. Once the phase mask is updated, we repeat this process, cycling over each plane in turn until the output field converges. Viewed from the perspective of wavefront shaping, our self-configuring MPLC can be understood as a ‘multi-plane wavefront shaper’, with the advantage that light can—in principle—be shaped more efficiently42, and multiple independent modes can be controlled simultaneously, as we show in what follows.

Figure 2 shows a flowchart depicting our in-situ MPLC design protocol. To begin the optimisation, the MPLC is illuminated with input field u. In the initial MPLC configuration, field u will flow through the optical system generating an output field v0 that typically has a low correlation with the target output field v. Here we represent u, v0 and v as column vectors—vectorised versions of the pixelated 2D input and output fields. The MPLC planes are indexed by integer m which takes values from 1 to M. We aim to calculate how to update the phase delays imparted by all pixels on plane m to improve the performance of the MPLC.

The propagation of light through the MPLC can be represented by

v0 = Tm · Dm · Hm · u, (1)

where matrix Hm is the TM linking the input field u to the field arriving at plane m within the MPLC, and matrix Tm is the TM linking the field leaving plane m to the output field v0. Here Dm is a diagonal matrix representing how the phase of the light field flowing through the MPLC is modified by plane m. We first measure the TM Tm. We sequentially display a set of P plane-waves (we also tested Hadamard and 2D discrete cosine functions—see Supplementary Section S7). For each test mode, the corresponding transmitted field arriving at the output (camera) plane is measured holographically. The transmitted fields are vectorised and stacked as columns of T0m —here the prime indicating that the input basis of T0m is different from the pixel input basis of Tm shown in Eq. (1).

Once measured, T0m can be used to calculate the complex spatial filter, sm, that, if placed at plane m inside the MPLC, would convert the field incident on plane m into the field that will subsequently evolve into v at the output:

sm = (T0m)−1 · v. (2)

Here sm is a column vector expressing complex coefficients in terms of the plane-wave basis used to measure T0m. Experimentally we take (T0m)−1 = (T0m)y, under the assumption that T0m is unitary (see Methods). Importantly, sm naturally takes into account the unknown shape of the field incident on plane m inside the MPLC (u0m = Hm · u). As each MPLC plane can only modify the phase of the light flowing through it (and our aim is to perform a lossless unitary transform using a cascade of phase-only masks), we take the argument of sm to obtain the phase mask function ϕm:

ϕm = arg R(sm), (3)

where matrix R transforms the representation of sm from the plane-wave basis to the micro-mirror pixel basis (see Methods). Plane m is updated to ϕm, thus improving the MPLC design. This phase mask update constitutes one iteration of our algorithm. We iterate through all M phase masks in this way, and then continue cycling over the planes until the design converges. More than one update of each plane is typically necessary, since when looping back to plane m, the phase functions of the surrounding planes have changed, and so further updating plane m can continue to improve the design. Convergence is designated by the change to the phase planes falling below a threshold level, or no further improvement in the fidelity of the output field being observed.

We now expand this design concept to handle N input modes simultaneously—an example of an MPLC transforming N=3 modes is given in Fig. 1c. We label input and output mode pairs with un and vn respectively, where n indexes the mode pairs from 1 to N. To calculate the updated phase profile of each plane, we illuminate the MPLC with the N input modes in turn, and in each case measure the TM from plane m to the output plane. For example, T0m,n is the TM measured from plane m when the MPLC is illuminated with input mode un. We calculate a mode pair-dependent set of complex filters sn,m = (T0m,n)−1 · vn, and the updated phase function to be displayed on plane m is given by

ϕm = arg R(∑n sn,m). (4)

Here the sum over the set of N complex filters sn,m serves to find a phase function that multiplexes the action of the phase plane to simultaneously improve the mapping of each input mode to its respective output mode. Supplementary Section S11 provides a high level flowchart detailing all of the steps in our self-configuring MPLC routine.

Our framework can be understood in the context of the wavefront matching method29: a coordinate descent based inverse design scheme that is often used to numerically design MPLCs23. See, for example, ref. 30 (Supplementary information) for a derivation of the wavefront matching method applied to MPLC design. As in our in-situ MPLC optimisation algorithm, the wavefront matching method also relies on determining the complex spatial filters sn,m to calculate how to improve the phase profile on plane m. In the wavefront matching method, sn,m is found by forward propagating input mode u to plane m, backward propagating the target mode vn to plane m, and comparing these fields—which represents an efficient adjoint optimisation approach.

It is, in principle, possible to physically achieve both the forward and backward propagation steps necessary for the wavefront matching method to adjointly optimise an MPLC—an approach that is a physical analogue of the error back-propagation algorithm used to train neural networks43. For example, ref. 44 has explored this concept through simulations. Indeed, there is much interest in such approaches for in-situ training of physical neural networks37. However, our aim here is to avoid the substantial additional complexity and alignment challenges associated with constructing an optical system capable of sending shaped light in both directions (akin to arranging two digital optical phase conjugation systems back to back45,46) and accurately holographically imaging the planes inside the MPLC.

In our scheme, light is transmitted only in the forward direction, and we use TM measurement to recover the complex spatial filters sn,m. Reliance on TM naturally entails making many measurements to calculate each new updated phase function, so our protocol does not classify as an adjoint method. However, since our approach draws inspiration from the wavefront matching method, large changes to the phase mask profiles can be made on each mask update, resulting in optimisation in relatively few mask update cycles. The convergence properties of our algorithm also follow those of the wavefront matching method. In Supplementary Section S2, we show simulations comparing the performance of our self-configured MPLC design method to that achievable via offline design using the wavefront matching method. We find that when the number of optimisation parameters (i.e., M×P) is held constant, the two approaches give the same theoretical performance.

Fast-switching MPLC platform
To experimentally implement our in-situ MPLC optimisation routine, we introduce a fast-switching MPLC platform, allowing millions of holographic TM measurements to be made on a practical timescale. We employ a new type of SLM known as a phase-only light modulator (PLM)38,47–49, shown schematically in Fig. 1a. PLMs are MEMS SLMs consisting of mega-pixel arrays of micro-mirrors. Each micro-mirror can be pistoned vertically with 4-bit precision (i.e., to one of 16 mirror heights), thus controlling the phase of reflected light. Micro-mirror response time is less than 50 μs, resulting in fundamental switching rates of ∼20 kHz—although the currently available development models are limited to continuous modulation rates of 1.44 kHz by their control electronics. The pixel pitch of our PLM model is 10.8 μm, with a pixel fill factor of 94%. Thus it delivers high-efficiency beam shaping on-par with liquid crystal SLMs, and is compatible with the multiple reflections and zero-diffraction order beam shaping of an MPLC architecture. We recently showed how PLMs could be used for high-fidelity wavefront shaping through complex media, and developed bespoke C++ software to synchronise data transfer and continuously display holograms at up to 1.44 kHz50. Here we build on this work and program a fast-switching self-configuring PLM-based MPLC.

Figure 3a shows a schematic of our experimental setup, which is based on a Mach–Zehnder interferometer. A collimated laser beam is split into two paths. In the upper path, light first reflects from a liquid crystal SLM which is used to generate the input spatial modes un incident onto the MPLC. We construct a 4-plane MPLC using a mirror placed opposite the PLM chip (also see Fig. 1a). Four planes were selected in our proof-of-principle experiments as a trade-off between MPLC fidelity, efficiency and optimisation time—see Supplementary Section S6. The transmitted light is imaged onto a high-speed camera which is synchronised with the update cycle of the PLM. The image plane of the camera is located a few centimetres after the final MPLC plane. A reference beam takes the lower path of the interferometer, and is also imaged onto the camera enabling measurement of the fields transmitted through the MPLC via single-shot off-axis digital holography51.

Since our approach relies on making a large number of interferometric measurements, it is crucial to ensure that the phase drift between the two arms of the interferometer is stabilised within each mask update. Standard phase drift tracking methods (e.g., ref. 52) cannot be directly applied as our scheme relies on the consecutive measurement of TMs with differing MPLC input modes. Therefore, we develop a specific phase stabilisation protocol which is detailed in the Methods. We found this was critical to obtain high-fidelity results.

Prior to commencing an optimisation, it is necessary to define the area of the PLM corresponding to each phase mask. It is enough to roughly estimate the centre of each reflection. No knowledge of the distance between the phase masks, the distance from the last plane to the output camera, or the axial position of the first plane with respect to the incident beams is required. Indeed, our approach is not only limited to free-space MPLCs, but is compatible with any mode-mixing elements placed between the planes. In our experiments, we initialise the phase masks by uniformly setting the phase of all pixels to 0 rad, although any choice of phase mask initialisation can be used—see Supplementary Section S5 for more discussion on the effect of different initialisations. The number of plane-waves used to sample each TM sets the effective resolution of the corresponding phase mask. Here we tested between P=4096–8100 plane-waves, with the range of plane-wave k-vectors chosen to ensure uniform sampling and no aliasing (see Methods).

Arbitrary field reshaping and universal mode sorting
To test our in-situ MPLC optimisation approach, we first task it with reshaping a single input field to a new target output field. Such reshaping has previously been used, for example, to efficiently generate arbitrary images53 or couple arbitrarily shaped optical fields into single-mode fibres54. Figure 1b shows the mapping of a speckle pattern into a Laguerre–Gaussian (LG11) mode. We plot examples of the output field at different stages in the optimisation process, and observe that after 40 mask updates, the fidelity of the output mode reaches 0.95. Supplementary Movie 1 shows the output field as a function of mask update number throughout the optimisation process.

Next, we optimise the MPLC to simultaneously reshape three orthogonal input speckle fields into Hermite–Gaussian modes: HG13, HG22, and HG31, as shown in Fig. 1c. Orthogonal speckles are generated as described in ref. 30. Here slightly lower fidelities of 0.87, 0.92, 0.87 are achieved, respectively, due to the increased complexity of the transformation. The fidelity could potentially be further boosted by increasing the number of test modes used in the measurement of each TM, thus increasing the resolution of the phase masks (see Supplementary Section S2). Supplementary Section S1 shows the fidelity of the output as a function of mask update number for Fig. 1b, c—we see that the MPLC designs have converged after ∼20 mask updates. Supplementary Section S10 shows the repeatability of our design process. Supplementary Movie 2 shows the three different output fields generated when the MPLC is illuminated with the three different input fields, as a function of mask update number throughout the optimisation process.

We now turn our attention to spatial mode sorting: redirecting the energy carried by a set of orthogonal input spatial modes to separate locations across a transverse plane at the output. Spatial mode sorters have a variety of future applications in the fields of imaging and optical communications6,20,55. In Fig. 4, top row, we demonstrate the optimisation of a 10-mode HG sorter. Figure 4a shows examples of the light from individual spatially overlapping input modes being redirected to separate output channels. Following ref. 23, we arrange the output channels in a triangular lattice, as this configuration has been shown to lead to an efficient HG mode sorter design. All ten input HG modes are depicted in Fig. 4b in the arrangement they will be sorted into. Figure 4c shows the incoherent sum of the output intensities recorded when the HG mode sorter is illuminated with each mode in turn.

Figure 4d shows the mean total cross-talk throughout the optimisation process. Here we compare our experiment to a simulation of an idealised system. We see a similar rate of convergence and generally good agreement between our simulations and experiments. The simulated mean total cross-talk plateaus at a lower value than in our experiments. This is because the simulation represents the best possible case in which the phase function of each mask is continuous (rather than discretised into 16 phase levels as in our experiment), the SLM is 100% efficient, and there is no experimental noise or residual phase drift in the measurements. See Supplementary Section S12 for further discussion on the experimental factors limiting our approach, and possibilities for improvement in future. Figure 4e shows the experimentally measured cross-talk matrix, with an average cross-talk of −19 dB per channel (i.e., the average value of the off-diagonal elements).

Figure 4, bottom row, shows equivalent results for the optimisation of a 7-mode orthogonal speckle sorter30—highlighting the universal nature of the spatial transformations that our approach can handle. Here the output spots are arranged into a hexagonal grid. Speckle sorters are examples of arbitrary basis rotations, and have applications in unscrambling light that has propagated through scattering media7. In this case, the average cross-talk is −17 dB per channel—higher than the cross-talk for HG mode sorting, since no efficient low plane count MPLC design exists for arbitrary speckle sorting. Supplementary Section S3 shows that reducing the number of sorted speckle fields to N=5 further decreases the cross-talk to −18 dB per channel. Figure 3b shows experimental examples of the phase masks displayed throughout the speckle mode sorter optimisation process.

Optimisation timescales
An important aspect of our approach is the time it takes to converge. In our proof-of-principle implementation, the total number of MPLC configurations that need to be tested scales according to O(P N M C), where P is the number of samples per TM, N is the number of input modes, M is the number of planes, and C is the number of cycles of each plane. More specifically, the time to measure and process the data from a single TM, tTM, is given by

tTM ≈ (1 + rdrift) P / f + dTM, (5)

where rdrift is the fraction of extra measurements needed for phase drift tracking (see Methods), f is the SLM modulation rate and dTM is the time required for digital holography data processing (which depends upon the size of the region of interest of the camera and P). The overall MPLC optimisation time, topt, is given by

topt ≈ tTM (N + 1) + dmask MC, (6)

where dmask is the data processing time to create each mask update (which depends upon the size of the mask and P). The extra TM measurement is used for inter-TM phase drift tracking (see Methods).

A key advantage of our approach is that ∼P(N+1) MPLC configurations (i.e., thousands in this work) can be rapidly sampled without the need for any decision logic to redesign SLM holograms, since calculation of new MPLC patterns only happens at the point of mask update.

The 4-plane, 10-mode HG sorter shown in Fig. 4 used rdrift = 0.08 and a set of P=4096 plane-waves to measure each TM. In our software implementation, dTM ≈ 6 s and dmask ≈ 7 s. Here we operated the PLM at f=720 Hz, which is half of its maximum modulation rate, due to the limited frame-rate of our camera when capturing a larger field of view. This resulted in a TM measurement time of tTM ≈ 12 s, and so each mask update took ∼140 s. The total optimisation time for C=5 cycles was topt ≈ 47 min, which constituted 20 mask updates via the measurement of 220 TMs, achieved by sampling a total of ∼970,000 different MPLC configurations. Table 1 gives the optimisation times (column 11) of all of the self-configured MPLCs demonstrated in this work.

There is scope to substantially decrease these optimisation times in the future. For example, PLMs have a fundamental switching time lower than 50 μs, and models with frame-rates of up to f=10 kHz are currently under development48. In addition, the time required for digital holography data processing and phase mask calculation can be markedly reduced using parallelised routines and optimised libraries56, such that dTM and dmask become negligible. If coupled with higher frame-rate sensors, these improvements would reduce the timescale required to optimise the HG mode sorter we show here from topt ≈ 47 min to topt ≈ 88 s. Likewise, reshaping of a single input beam could be achieved in topt ≈ 9 s. Column 12 of Table 1 gives projected future optimisation times of all MPLC demonstrated here if using a next-generation PLM.

In addition to speeding up the PLM frame-rate, we expect it will also be possible to heavily reduce the number of measurements that need to be made. This could be achieved in multiple ways. For example, here we initialise the phase masks with a flat phase function, while if we have some knowledge of the MPLC geometry and are able to use this to coarsely align the system manually with a pre-designed set of phase masks, in-situ optimisation could be used to fine-tune the design. See, for example, refs. 7,57,58 for manual MPLC alignment protocols. Optimising the position of each phase mask has also been accomplished using genetic algorithms30,32. Combining our automated approach with these methods could reduce the number of mask update cycles C needed for the design process to converge—as explored in simulations in Supplementary Section S8.

Furthermore, here we have fully-sampled every TM, under the assumption that we have no knowledge about the transfer function of the optical system. However, we know the updated state of each phase mask throughout the optimisation process. Even assuming we have imprecise knowledge of the optical system—such as the geometry and the actual phase delays imparted by the phase masks—this knowledge could be made use of via the framework of compressive sensing59. For example, full TMs have been reconstructed using only ∼10% of the conventionally required number of measurements by exploiting sparsity priors in the TM structure60. Alternatively, adaptive optimisation schemes in place of TM measurement could potentially reduce sampling requirements, at the cost of more regularly updating the phase masks61,62. Indeed, our knowledge about the entire optical system steadily increases throughout the optimisation process, as we collect data on the response of the MPLC as a function of micro-mirror state. This information could be used to construct a physically accurate model of the system so that future MPLC designs can be conducted partially or wholly offline. Putting prior knowledge and measured data to good use to speed up optimisation times will be the focus of our future work.

Methods
Phase drift correction
Since our optimisation approach relies on making a large number of interferometric measurements, it is crucial to ensure that phase drift between the two arms of the interferometer is stabilised. Achieving this is not straightforward, as the optimisation relies on the consecutive measurement of TMs with different input modes. Therefore, we develop a specific phase stabilisation protocol, which is split into two steps: intra-TM and inter-TM phase drift correction.

Intra-TM phase drift correction refers to phase stabilisation within the measurement of a single TM. Here we use a conventional approach of interlacing TM measurements with a standard measurement. The global phase of this standard measurement tracks the phase drift as a function of time throughout the TM measurement. On compiling the TM, the global phase of each TM column is subsequently adjusted to negate the effect of phase drift. In our experiments, we insert an intra-TM drift measurement after every 12 measurements, which increases the total number of measurements by ∼8% (i.e., rdrift = 0.08). Given the typical modulation rate of f=720 Hz in our experiments, this meant a drift measurement was made at a rate of ∼65 Hz, which was much higher than the rate of path length drift between the arms of the interferometer in our case.

Inter-TM phase drift stabilisation corrects the global phase of each of the N TMs measured with different input modes: the nth TM from the mth plane being labelled T0m,n. To achieve this, after measuring the first N TMs with different input modes, we create a new input mode which is the sum of all N input modes. We transmit this new input mode through the MPLC system while the mth plane displays a plane-wave of index k. This results in a scattered field vkall arriving at the output camera (Cam2). This final measurement is related to the earlier TM measurements via

vkall = ∑n=1N eiθn vkn, (7)

where vkn is the kth column of TM T0m,n, and θn is the unknown global phase drift associated with the nth TM that we aim to recover—i.e. vkall is the sum of the previously measured vkn for all n, with each term weighted by the unknown phase drift. Equation (7) can be represented as the matrix equation

vkall = Vk · dk, (8)

where vkn form the nth column of matrix Vk, and eiθn is the nth element of column vector dk. To find the unknown phase drift terms, we rearrange Eqn. (8) to solve for dk:

dk = (Vk)−1 · vkall. (9)

We note that if the entire transmitted field is not captured, the columns of Vk are not orthogonal. In this case (Vk)−1 is given by the Moore–Penrose pseudoinverse of Vk.

In principle, dk should be independent of the choice of plane-wave (indexed by k) displayed on plane m for the drift calibration measurement. To improve the signal-to-noise ratio of inter-TM drift tracking, in our experiments we take the mean drift phase, averaged over all displayed plane-waves, such that the drift phase associated with the nth TM, θn, is given by

θn = arg [∑k dkn], (10)

where dkn is the nth element of dk. Using this approach, a mask update requires the measurement of N+1 TMs. See Supplementary Section S10 for examples of the typical measured levels of intra-TM phase drift in our experiments, and examples of the measurement of θn via our inter-TM phase drift correction process.

TM sampling
We typically sample the TM with a number of plane-waves that is lower than the number of pixels across each phase mask. Therefore, to ensure each phase profile is uniformly sampled in the plane-wave basis with no aliasing, the maximum transverse component of the plane-wave k-vector is given by

kmax = π√P / (p npix), (11)

where p is the micro-mirror pitch and npix is the number of micro-mirrors across one phase mask. For example, in the HG sorter (Fig. 4a–e), npix = 256 micro-mirrors wide, meaning the total number of pixels per plane is npix2 = 65536. Thus when sampling the TM with P=4096 plane-waves, the final phase masks have an equivalent resolution of √P × √P = 64×64 super-pixels, each of size npix/√P = 16 micro-mirrors (i.e., a patch of 4×4 micro-mirrors).

To recover the phase mask update function, ϕm, we use Eq. (3) (for N=1) or Eq. (4) (for N>1). Here the matrix R transforms from the plane-wave basis to the micro-mirror pixel basis. Each column of R is given by the plane-wave function displayed on plane m of the MPLC during TM measurement: exp [i(kx x + ky y)], where here x and y denote the lateral Cartesian coordinates of the micro-mirrors, and kx and ky specify components of the k-vector of each plane-wave (also noting that |k|=2π/λ).

Peer review rounds
[{"round": 1, "reviewer_comments": {"reviewer_1": "The authors report a self‐configuring free‐space multi‐plane light converter (MPLC) that integrates in-situ transmission-matrix (TM) measurement with a fast MEMS-based, phase-only light modulator. By relying solely on forward propagation through the device, their iterative scheme updates each phase plane to map arbitrary input mode sets (speckle, HG, LG) onto target output positions or modes with high fidelity. They present a thorough analysis of optimization timescales, introduce phase-drift correction protocols, and suggest clear avenues for improving efficiency and speed. Before accepted by Nature Communications, the following points should be addressed:\n1.Are the fidelity and phase map of the final output pattern consistent when the optimization is repeated multiple times in the same configuration? How strongly does the algorithm depend on the initial phase (e.g., all zeros)?\n2.A flowchart illustrating the sequence of multi-mode TM measurements, intra-/inter-TM drift corrections, and phase-plane updates would greatly aid comprehension; It is not obvious whether all input modes (in Fig. 1c) are processed simultaneously or in serial loops, please also indicate it in the flowchart\n3.The choice of four planes seems arbitrary. How would performance, speed, or device complexity change if you used fewer or more planes?\n4.You employ plane-wave, Hadamard, and 2D discrete cosine functions bases to sample each TM. How does this choice influence convergence speed and final fidelity? Could adaptive or sparsity-driven bases reduce measurement overhead?\n5.The current demonstrations handle up to ten modes. How do measurement time, TM inversion cost, and convergence scale for N>100? What are the primary algorithmic or experimental bottlenecks when orthogonality is harder to maintain?\n6.How robust is your two-stage drift-correction protocol to real-world temperature fluctuations or vibrations? Must the optimisation restart if drift exceeds threshold, or can it continue seamlessly?\n7.All experiments use a single wavelength. What modifications to the optimization workflow or hardware would be required to handle broadband or multi-wavelength inputs?\n8.Compared to traditional free-space MPLC implementations, what are the relative advantages or drawbacks in terms of power consumption, system footprint, and component cost?\n9.You propose that gain-coated mirrors or pixel-level design optimisations could boost end-to-end efficiency to 20–40%. Please provide supporting simulations or prototype measurements to substantiate these claims.", "reviewer_2": "In this manuscript, the authors have shown an experimental implementation of a fast, self-configuring, multi-plane light converter (MPLC). Using a MEMS-based SLM for wavefront manipulation, the authors time-efficiently measured the transmission matrices between each plane of the MPLC and the output ‘plane’ to implement a live version of the wavefront matching algorithm. This method, in principle, should allow them to effectively account for any misalignments, aberrations, and other linear characteristics of the physical system. The techniques used in the work are well documented. This work is original, novel, highly applicable, and extremely relevant to the field, making it potentially suitable for publication in Nature Communications.\nHowever, there is one minor concern that needs to be addressed before I can give my final recommendation. The authors motivated this work by highlighting the difficulties in properly characterising the physical system of the MPLC that leads to a loss of real-world performance. In this regard, does a TM-based MPLC implementation improve the real-world performance achieved by MPLCs relying on physical models? How do the results (cross-talks and fidelities) presented in this work compare to prior art, and what are they (if at all) limited by?"}, "author_response": "Replytoreviewers’commentsonSelf-configuringhigh-speedmulti-planelight\nconversion by José C. A. Rocha et al.\nWe would like to thank the reviewers for carefully reading our manuscript, and for the time they\nhave dedicated to evaluating our work. We have incorporated their suggestions into the updated\nmanuscript and prepared 8 new supplementary sections, which we believe has strengthened our\npaper. Below,werespondtothereviewers’commentsonapoint-by-pointbasisinpurple.\nReviewer1:\n1.0: The authors report a self configuring free space multi plane light converter (MPLC) that inte-\ngrates in-situ transmission-matrix (TM) measurement with a fast MEMS-based, phase-only light\nmodulator. By relying solely on forward propagation through the device, their iterative scheme\nupdates each phase plane to map arbitrary input mode sets (speckle, HG, LG) onto target output\npositionsormodeswithhighfidelity. Theypresentathoroughanalysisofoptimizationtimescales,\nintroduce phase-drift correction protocols, and suggest clear avenues for improving efficiency and\nspeed. BeforeacceptedbyNatureCommunications,thefollowingpointsshouldbeaddressed:\nResponse: Wethankthereviewerfortheirencouragingassessmentofourpaper.\n1.1: Are the fidelity and phase map of the final output pattern consistent when the optimization is\nrepeatedmultipletimesinthesameconfiguration?\nResponse: We observe that when the optimization is repeated sequentially under similar exper-\nimental conditions, it yields similar mask designs, which give a nearly identical level of perfor-\nmance. To demonstrate this, we have added a new Supplementary Section §10, where we show a\nside-by-sidecomparisonofthefinalmaskdesignsandthesystemperformancethroughouttheop-\ntimisation process, when experimentally undertaking the same MPLC design twice (3 orthogonal\nspeckles mapped to 3 HG modes). In both cases, we started from the same initial conditions of\nuniformlyflatphasemasks. Weseethatwhilethefinalphasemasksarenotidentical,verysimilar\nfeatures are shared across the two designs. We put the differences in the design down to the effect\nofmeasurementnoiseandanysubtlechangesinsystemalignmentleadingtheoptimisationroutine\ndown a different design pathway. Despite the small differences in the final mask designs, we find\nthat the system performance (here quantified as output mode fidelity) are very similar throughout\ntheentiredesignprocess.\nWenowrefertothisnewsupplementarysectioninthe“Arbitraryfieldreshapinganduniversal\nmodesorting\"sectionofthemaintext,wherewestate:\n“SI§10showstherepeatabilityofourdesignprocess\".\n1.2: Howstronglydoesthealgorithmdependontheinitialphase(e.g.,allzeros)?\nResponse: Wefindthatstartingtheoptimisationprocessfromdifferentinitialsetsofphasemasks\ndoes lead to differently optimised sets of phase masks which can vary in performance. This be-\nhaviour is similar to the conventional wavefront-matching algorithm when started from different\ninitial conditions. We have added a new supplementary section §5, where we simulate the level of\nvariation in the cross-talk and efficiency of the 10-mode HG mode sorter when starting from flat\nphase masks and 40 different randomly chosen phase mask sets. In this case we find that MPLC\ndesigns initialised with flat phase consistently outperform MPLC designs initialised with random\nphase masks: while cross-talk is similar when the algorithm is initialised in either way, the design\nefficiency is improved by ∼ 25% when using a flat phase as an initialisation. We also find that the\nperformance of the randomly initialised phase masks are relatively independent of the particular\nrandomlydrawnstartingmasks.\nExperimentally, that there is a technical advantage to starting the optimisation process from\nflat phase masks rather than random phase patterns. Using flat phase masks means we can di-\nrect the majority of the transmitted light onto camera from the start of the optimisation process.\nIn contrast, starting from random phase masks has the tendency to scatter light out of the opti-\ncal system where it is not measured by the camera, which lowers the signal to noise ratio of the\nmeasurements, at least for the first few mask updates. Flat phase masks also represent the most\nwavelength-independentinitialguess,reducingtheriskofconvergingtoahighlystructured,more\nwavelength-sensitivesolutionthatisalsomoresusceptibletosystemdrift.\nWe now refer to this new supplementary section in the “Fast switching MPLC platform\" sec-\ntionofthemaintext,wherewestate:\n“Inourexperiments,weinitialisethephasemasksbyuniformlysettingthephaseofallpixelsto0\nrad, although any choice of phase mask initialisation can be used – see SI §5 for more discussion\nontheeffectofdifferentinitialisations\".\n1.3: A flowchart illustrating the sequence of multi-mode TM measurements, intra-/inter-TM drift\ncorrections, and phase-plane updates would greatly aid comprehension; It is not obvious whether\nall input modes (in Fig. 1c) are processed simultaneously or in serial loops, please also indicate it\nintheflowchart.\nResponse: Hereweagreewiththereviewerthataflowchartwouldclarifyouralgorithm. Wehave\nnow added a flowchart to a new supplementary section §11 (Fig. S12), which details all of the op-\ntimisationsteps. Toanswerthespecificquestion: Foraparticularmaskupdate,theTMassociated\nwith each input mode is measured sequentially, and the results are processed together to give a\nsinglemaskupdateaveragedoverallinputmodes.\nWenowrefertothisnewsupplementarysectioninthe“In-situMPLCoptimisationalgorithm\"\nsectionofthemaintextwherewestate:\n“SI §11 provides a higher-level flowchart detailing all of the steps in our self-configuring MPLC\nroutine.\"\n1.4: The choice of four planes seems arbitrary. How would performance, speed, or device com-\nplexitychangeifyouusedfewerormoreplanes?\nResponse: Wechosefourplanesinthisstudybasedonatrade-offbetweenthefollowingcriteria:\n(i)Minimisingtheoptimisationtime(whichscaleslinearlywiththenumberofplanes);\n(ii)Achievingrelativelylowcross-talk(whichimproveswiththenumberofplanes);\n(iii)Maximisingtheoverallefficiency(whichdecreaseswiththenumberofplanesduetoreflection\nlosses, despite the fact that the design efficiency itself gradually increases with the number of\nplanes).\nWenowdiscussthistrade-offinanewsupplementarysection§6,wherewealsopresentsimula-\ntionsoftheHGmodesorterandspecklemodesorterperformance(cross-talkanddesignefficiency)\nwhen varying the number of planes from 2 to 10. We show that although there is improvement in\nMPLC fidelity beyond 4 planes, it is relatively gradual, thus requiring many more planes to sig-\nnificantly improve the MPLC performance. More generally, the number of planes M needed to\nefficientlyenactanarbitrarytransformonN modestypicallyscaleslinearlywithN [1].\nWe now comment on this in the “Fast-switching MPLC platform\" section of the main paper\nwherewestate:\n“We construct a 4-plane MPLC using a mirror placed opposite the PLM chip (also see Fig. 1(a)).\nFour planes were selected in our proof-of-principle experiments as a trade-off between MPLC fi-\ndelity,efficiencyandoptimisationtime–seeSI§6.\".\n1.5: You employ plane-wave, Hadamard, and 2D discrete cosine functions bases to sample each\nTM.Howdoesthischoiceinfluenceconvergencespeedandfinalfidelity?\nResponse: We found that the plane-wave basis consistently led to the best performance in terms\nof convergence speed, crosstalk and efficiency. This is because, when measuring a TM with\nfewer measurements than there are PLM pixels on each plane, the plane-wave basis converges\nto smoothly varying phase masks – which have also been demonstrated to be optimal for wave-\nfront shaping within forward scattering scenarios such as ours [2, 3]. In contrast, the Hadamard\nbasis will converge to coarsely pixellated phase mask designs which have sharp edges that will\nscatterlightathighanglesoutoftheopticalsystem.\nFurthermore,thereisacomplicationwhenattemptingtousethediscretecosinebasis: Theba-\nsis vectors of the plane-wave and Hadamard basis are functions in which only the phase spatially\nvaries, meaning they can be accurately represented using a phase-only spatial light modulator.\nConversely,thebasisvectorsofthediscretecosinebasisspatiallyvaryinboththeiramplitudeand\nphase, and as such cannot be accurately represented using a phase-only spatial light modulator in\nalosslessmanner. Thisleadstosub-optimalperformanceofthediscretecosinebasis.\nWe have now added a new supplementary section §7 in which we show in simulation the rel-\native performances of the optimisation algorithm when using different bases to sample the TM,\nand we further discuss their respective differences. We refer to this supplementary section in the\n“In-situMPLCoptimisationalgorithm\"sectionofthemainpaperwherewestate:\n“wealsotestedHadamardand2Ddiscretecosinefunctions–seeSI§7\".\n1.6: Couldadaptiveorsparsity-drivenbasesreducemeasurementoverhead?\nResponse: We think that adaptive or sparsity-driven approaches are promising directions for re-\nducing measurement overhead. In the ‘Optimisation timescales’ section of the initial version of\nthe main paper, we raised the possibility of using compressive sensing, which can take advantage\nofsparsitypriorsintheTM,wherewestated:\n“Evenassumingwehaveimpreciseknowledgeoftheopticalsystem–suchasthegeometryandthe\nactual phase delays imparted by the phase masks – this knowledge could be made use of via, for\nexample, the framework of compressive sensing [59, 60]. This approach has the potential to sub-\nstantiallyreducethenumberofsequentialmeasurementsneededtoreliablyconstructeachTM.\"\nForexample,inmainpaperref.[60]ourgrouphaspreviouslyshownthatTMscanbereconstructed\nusingaslittleas5-10%ofthefully-samplednumberofmeasurementsbyrelyingonsparsitypriors.\nAs the reviewer mentions, even without using prior knowledge, there is also an opportunity\nto speed up the design process via adaptive approaches. For example, at each mask update step,\nour presented method measures the TM which is then used to find the mask m update sn for a\nm\nparticular input mode n. In principle, it should be possible to find a good approximation to sn\nm\nwithfewermeasurementsbyadaptivelyoptimisingthephasemasktogeneratethetargetnth output\nmode using methods presented in e.g., refs. [4, 5]. The drawback of these approaches is that the\nphase masks must be adaptively updated more regularly throughout the optimisation procedure\nthan our present approach. Regularly updating the phase masks is not presently compatible with\nfastPLMoperation(asupdatedphasemaskscanonlybeloadedatatrateofupto60Hz).\nWehavenowupdatedthepapertoincludethesepointswherewestate:\n“Furthermore,herewehavefully-sampledeveryTM,undertheassumptionthatwehavenoknowl-\nedgeaboutthetransferfunctionoftheopticalsystem. However,weknowtheupdatedstateofeach\nphase mask throughout the optimisation process. Even assuming we have imprecise knowledge\nof the optical system – such as the geometry and the actual phase delays imparted by the phase\nmasks – this knowledge could be made use of via, for example, the framework of compressive\nsensing [59]. For example, full TMs have been reconstructed using only ∼10% of the conven-\ntionally required number of measurements by exploiting sparsity priors in the TM structure [60].\nAlternatively,adaptiveoptimisationschemesinplaceofTMmeasurementcouldpotentiallyreduce\nsamplingrequirements,atthecostofmoreregularlyupdatingthephasemasks[61-62]\".\nIntegrating such compressive or adaptive strategies would require substantial changes to both the\nmeasurementprotocolandreconstructionalgorithm. Furthermore,ourcurrentmethoddoesnotas-\nsumepriorknowledgeaboutthemodestructureorsystemsparsity,andassuchrepresentsthemost\ngeneralapproach. Nonetheless,weagreemeasurementreductionstrategiesareworthexploringin\nfuturework.\n1.7: Thecurrentdemonstrationshandleuptotenmodes. Howdomeasurementtime,TMinversion\ncost,andconvergencescaleforN > 100?\nResponse: Weinvestigatedthis,andhavenowincludedasupplementarysection§9demonstrating,\ninsimulation,theuseofouralgorithmtodesignanMPLCthatsorts105Hermite-Gaussianmodes.\nAkin to the conventional wavefront-matching algorithm, scaling to higher mode counts increases\nthe number of required mask updates for convergence. In this 105-mode design, we found that\napproximately 80 mask updates were required for reasonable levels of crosstalk and efficiency,\nwhich requires over 8,000 TM measurements to reach that point. In the current experimental\nimplementation, the total runtime is roughly evenly split between time in hardware (hologram\ndisplay–atthePLMrateof720Hzor1.44kHz)andprocessingtimeinsoftware(mainlyspentin\ndigital holography). At a hologram rate of 1.44kHz, we calculate it would take just over 14 hours\nin hardware for all TM measurements alone. Looking ahead, in the future development of PLM\ntechnology,ahologramrateoff = 10kHzcouldreducethistimetojustover2hours.\nComputational cost of inverting a single TM (i.e., taking its complex conjugate transpose) is\nthe same for both the ten-mode and 105-mode sorter, provided that the camera resolution and the\nnumberofbasisvectorsusedtosampletheTMarefixed,yetthenumberofTMswearerequiredto\ninvert increases linearly with the number of input modes – i.e., for 105 modes, total TM inversion\ncost is a factor of ten larger than the case for our ten mode sorter. That said, as we mention in\nthe ‘Optimisation timescales’ section of the main paper, many of the software routines could be\noptimised benefiting from the highly parallel nature of many of those steps, and with larger mode\ncounts,theseoptimisationsbecomeincreasinglyworthwhile.\nOn the same note, the above analysis assumes that we fully sample each TM. As discussed in\nresponsetoreviewerquestion1.6above,smartersamplingstrategies,suchascompressivesensing\nbased sparsity-driven approaches, would become increasingly advantageous at this scale. Indeed\nsuchstrategiesareneededtorenderself-configurationpracticalforlargemodecounts.\nWenowrefertothenewsupplementarysectionandcommentonthescalingforlargenumbers\nofmodesinthe‘Discussionandconclusions’sectionofthemainpaper,wherewestate:\nAsourapproachmirrorsthewavefrontmatchingmethod,itcan,inprinciple,bescaleduptohan-\ndlemoremodes–seeSI§9. However,optimisationtimescaleslinearlywiththenumbersofmodes\nand planes in our present algorithm, leading to lengthy optimisation times for large mode counts.\nFurthermore, some applications require larger sets of training pairs, potentially numbering in the\nthousands [37]. Here alternative in-situ optimisation algorithms may be employed – see ref. [66]\nfor an example applied to optimise a single phase plane. In these cases, new strategies to reduce\nthemeasurementoverheadofourscheme,assketchedoutabove,wouldbecomeessentialtorender\nself-configurationpractical.\n1.8: Whataretheprimaryalgorithmicorexperimentalbottleneckswhenorthogonalityisharderto\nmaintain?\nResponse: We now address this point in the ‘Discussion and conclusions’ section of the main\npaper,wherewehaveadded:\n“We note that an MPLC preserves the inner product of any two input modes at the output. There-\nfore, as long as this condition is maintained in the specified MPLC mode transformation, our\nself-configuring algorithm is compatible with non-orthogonal input and output mode sets. When\ntheinnerproductbetweeninputandoutputmodesetsinnotmatched(e.g.,ifattemptingtospatially\nsort non-orthogonal inputs), then it is still possible to optimise an MPLC using our approach, but\nthedesignefficiencywillbereducedinproportiontotheoverlapbetweentheinputs[6].\"\n1.9: How robust is your two-stage drift-correction protocol to real-world temperature fluctuations\nor vibrations? Must the optimisation restart if drift exceeds threshold, or can it continue seam-\nlessly?\nResponse: Ourphasedriftcorrectionprotocolhasnomaximumphasedriftlimit–itcancontinue\nseamlessly throughout any range of global phase drift occurring between the reference arm and\nthe beam transmitted through the MPLC. That said, there is a limit to the rate of phase drift we\ncan accurately account for. In our current implementation we measure the relative drift at a rate\nof 60Hz. This means we assume that the phase drift occurs on a timescale slower than 1/60s.\nIn our experiments (on a floated optical table) this condition was easily satisfied. We have now\nadded a new supplementary section §10 demonstrating the typical drift correction under different\nlab conditions. If phase drift fluctuations may occur more rapidly, the drift measurement rate can\nbe increased up to a maximum of half of the modulator rate (i.e., 720Hz for our current PLM\noperating at 1.44kHz.) at the expense of roughly doubling the total optimisation time (due to all\noftheextradriftcorrectmeasurements).\nFor real-world operation, vibrations may also cause changes to laser pointing stability. If laser\npointingdirectionchangesappreciablywithinthemeasurementofasingleTMmeasurement,\nthen\nthis may introduce substantial errors in the TM. However, when the self-configuring algorithm is\nrunning, then it will self-correct (in the following mask updates) for errors due to isolated vibra-\ntions causing an erroneous measurement of an individual TM. The main effect of these influences\nwillbetoslowconvergencetime. Forexample,wefoundthattheopticaltablecanbeaccidentally\nbumped and the algorithm will automatically compensate for the resulting measurement errors.\nThecombinationofhigh-frequencyglobalphasedriftcorrectionandself-configurationmeansthat\nthe system is highly resilient to external perturbations. To further improve the resilience to pertur-\nbations,weenvisionthatinsteadofmeasuringtheTMoneachmaskupdate,adaptiveoptimisation\nof each mask update would naturally self-correct for isolated external perturbations [4, 5] – an\napproachalsomentionedinanswertoreviewerquestion1.6.\nWe now refer to this new supplementary section §10 in the ‘Phase drift correction’ subsection\ninthe‘Methods’sectionofthemainpaper,wherewehaveadded:\n“SeeSI§10forexamplesofthetypicalmeasuredlevelsofintra-TMphasedriftinourexperiments,\nandexamplesofthemeasurementofθ viaourinter-TMphasedriftcorrectionprocess.\"\nn\n1.10: All experiments use a single wavelength. What modifications to the optimization workflow\norhardwarewouldberequiredtohandlebroadbandormulti-wavelengthinputs?\nResponse: While our experiments were performed at a single wavelength, the underlying MPLC\ndesignis,similarlytotheconventionalwavefront-matchingalgorithm,compatiblewithbroadband\noperation. Thiscouldbeachievedeitherbymeasuringthesystemresponseatmultiplewavelengths\nsequentially (at the cost of longer acquisition time) or simultaneously using a broadband source,\nprovidedtheinterferometerarmsarewellpath-lengthmatched. Inthatcase,theexistingworkflow\ncouldberetained,thoughinitialalignmentbecomesmorechallenging.\nNo fundamental changes would be needed to the algorithm itself to handle multi-wavelength\ninputs – the measurement of each mode at each new wavelength can essentially be treated as a\ndifferent input mode with appropriate wavelength re-scaling when the mask update is calculated\nviamainpaperEqn.4.\nWe have now added a paragraph to the ‘Discussion and conclusion’ section of the main paper\nwherewediscussthisprospect:\n“In principle, it should be possible to extend in-situ MPLC optimisation to handle multicolour\ninputs using a tunable laser source that would enable input modes of different wavelengths to be\ntransmittedthroughtheMPLC.Inthiscasetheresultingwavelength-dependentcomplexfilterssn\nm\nshould be appropriately rescaled when calculating the mask updates. The maximum phase shift\nthatcanbeappliedbyaPLM(orliquidcrystalSLM)iswavelengthdependent,sothisshouldalso\nbeaccountedforinthedesignalgorithm.\".\n1.11: Comparedtotraditionalfree-spaceMPLCimplementations,whataretherelativeadvantages\nordrawbacksintermsofpowerconsumption,systemfootprint,andcomponentcost?\nResponse: We first comment on a comparison with alternative reconfigurable MPLC platforms.\nThesystemfootprintandpowerconsumptionofourPLM-basedMPLCaresimilartoconventional\nliquid crystal SLM-based MPLC setups. In both cases, the active display requires a power supply\nwith voltages continuously supplied to each pixel of the display. In our implementation, compo-\nnentcostsarealsocomparable. Atpresent,ourtrialmodelofTIPLMisconsiderablycheaperthan\nconventional LCoS SLMs (roughly a factor of ten lower in cost), although this may change when\nfully packaged versions are rolled out. In the future PLMs may benefit from economies of scale,\nsince they have been developed by TI for next generation mass-market display applications. Our\nalgorithmwould–inprinciple–worksimilarlywithaliquidcrystalSLM,thoughtheoptimisation\nwouldbesignificantlyslowerwhenusingstandard60Hzdevices.\nEvidently, if the masks never needed to change, the most energy efficient option would be\nto use fixed phase masks, which once fabricated draw no power. Fixed lithographically etched\nreflection-basedMPLCscanbefabricatedwithextremelyhighreflectivityandcanbemadehighly\ncompact. For example some of the authors recently demonstrated a 45-mode, 14-plane reflective\nMPLC occupying a volume of 2×4.8×1mm [7]. Research is ongoing into further shrinking\nMPLCs using direct laser writing. For example, a 2-mode transmissive device occupying a vol-\nume of 50×50×50µm has been demonstrated [8]. This is highly promising, however at this\nscaleitischallengingtomaintainhighfidelityandhighmodecapacityoperation.\nWe have now added a statement to the ‘Discussion and conclusions’ section of the main paper\ndiscussingtheabove:\n“More generally, we note that while reconfigurable MPLCs are highly versatile, they come at\nthe expense of lower energy efficiency and a larger system footprint than passive fixed MPLCs –\nwhichhaverecentlybeenrealisedatmillimetricscales[7]andhavepotentialtobefurtherminia-\nturised[8]\".\n1.12: Youproposethatgain-coatedmirrorsorpixel-leveldesignoptimisationscouldboostend-to-\nend efficiency to 20%-40%. Please provide supporting simulations or prototype measurements to\nsubstantiatetheseclaims.\nResponse: Here we sought to make some comments and projections about the potential effi-\nciency improvements in reconfigurable MPLCs. However since the reviewer recommends this\nclaim should be more strongly evidenced, we have removed this claim from the main paper. We\ndiscuss the potential for these improvements in the supplementary §4, where we also now give\nmoredetailabouthowtheseprojectionsweremade.\nInsummary,ourestimatesarebasedonasimpleexpressionusingthefollowingequation(also\nappliedinourpreviouswork[9]):\nη = (r d )M, (1)\nexp SLM SLM\nwhere η is the experimental efficiency of the physically realised MPLC, which is separated into\nexp\ntheproductoftwocontributions:\n- Reflection efficiency r – defined as the percentage of incident light reflected from the PLM\nSLM\nthatissenttothezerodiffractionorderwhenthePLMdisplaysaflatpattern;\n- Diffraction efficiency d – the percentage of the zero order light that can be controllably\nSLM\ndiffractedtothefirstorder,whenthePLMdisplaysaphaseramp. Thediffractionefficiencycanbe\nmodelled[10],anddecreasesforsteeperphaseramps,soherewechooseanavalueofd = 0.84\nSLM\nbasedonthetypicalspatialfrequenciesofthedisplayedphasepatternsinourMPLCdesigns.\nTo calculate an estimate for the overall experimental efficiency of the MPLC η , the reflection\nexp\nefficiency and diffraction efficiency terms are multiplied and the result taken to the power of the\nnumberofphasemasksM,whichaccountsforthecompoundingoflossesduetoreflectionmulti-\npletimes.\nIn our present work, we estimate that r ∼ 0.63. For our M = 4 plane MPLC, this yields\nSLM\nη = (0.63×0.84)4 ∼ 0.08,whichagreeswellwithourexperimentalmeasurements.\nexp\nOurpredictionsofpossiblefutureincreasesinefficiencyarebasedonprojectionsofhowmuchthe\nreflection efficiency might be improved in future systems. We assume that diffraction efficiency\nwillremainfixed. Wegivetwoexamples:\n(1) A near infra-red-optimised PLM is currently under development by TI (recently introduced\nin a presentation at SPIE Photonics West conference in January 2025 [10]), which is designed to\noperate across the wavelength range 730 < λ < 1630nm, and is expected to have an improved\nreflection efficiency of r ∼ 0.8. For an M = 4 plane MPLC, assuming a reflection efficiency\nSLM\nofr = 0.8suggestsanexperimentalefficiencyofη = (0.8×0.84)4 ∼ 0.2.\nSLM exp\n(2) Liquid crystal SLMs have undergone many decades of development, and in these cases the re-\nflectionefficiencyhasbeenimprovedusingnarrowbandanti-reflectioncoatingsofthecover-glass,\nand also dielectric backplane mirrors. For example, this has enabled the implementation of liquid\ncrystal SLM-based MPLCs of up to ten planes [11]. These features grant liquid crystal SLMs a\nreflection efficiency of up to r ∼ 0.95. For an M = 4 plane MPLC, assuming a reflection\nSLM\nefficiencyofr = 0.95suggestsanexperimentalefficiencyofη = (0.95×0.84)4 ∼ 0.4.\nSLM exp\nWe note that the near infra-red PLM mentioned above is not yet available for testing, so we\ncannotperformanyexperimentalvalidationofthisdevice. Meanwhile,highefficiencySLMshave\nbeenusedforMPLCsbyothergroups,andtheirreflectivityiswellestablished(e.g.,seethespecifi-\ncationfortheHoloeyePLUTO-2.1-NIR-113hasaquotedreflectivityof95%,aidedbyadielectric\nmirrorbackplaneandanti-reflectioncoatedcover-glass–seehttps://holoeye.com/products/spatial-\nlight-modulators/).\nReviewer2:\n2.0: In this manuscript, the authors have shown an experimental implementation of a fast, self-\nconfiguring, multi-plane light converter (MPLC). Using a MEMS-based SLM for wavefront ma-\nnipulation, the authors time-efficiently measured the transmission matrices between each plane of\ntheMPLCandtheoutput“plane”toimplementaliveversionofthewavefrontmatchingalgorithm.\nThis method, in principle, should allow them to effectively account for any misalignments, aber-\nrations, and other linear characteristics of the physical system. The techniques used in the work\narewelldocumented. Thisworkisoriginal,novel,highlyapplicable,andextremelyrelevanttothe\nfield,makingitpotentiallysuitableforpublicationinNatureCommunications.\nResponse: Wethankthereviewerfortheirfavourablecommentsaboutourwork.\n2.1: However, there is one minor concern that needs to be addressed before I can give my final\nrecommendation. Theauthorsmotivatedthisworkbyhighlightingthedifficultiesinproperlychar-\nacterising the physical system of the MPLC that leads to a loss of real-world performance. In this\nregard,doesaTM-basedMPLCimplementationimprovethereal-worldperformanceachievedby\nMPLCs relying on physical models? How do the results (cross-talks and fidelities) presented in\nthisworkcomparetopriorart?\nResponse: Toanswerthisquestionwefirstconsiderthedifficultiesfacedinexperimentallyimple-\nmentinganMPLC,whichfallintotwokeycategories: misalignmentsandaberrations.\nMisalignments: Theserefertotherelative3Dpositionsandorientationsoftheplaneswithrespect\nto one another and the optical axis of the input light. It is, in principle, possible to manually align\nan MPLC to correctly match these alignment degrees of freedom with the model used to design\ntheMPLC.Therefore,consideringonlymisalignments,wewouldexpectouralgorithmtomatcha\nperfectlymanuallyalignedMPLC(aslongasthesignal-to-noiseratioofthein-situmeasurements\\ishigh,andenoughTMsampleswereused–seeSI§2).\nAberrations: However, if the planes feature unknown aberrations – that are therefore not included\nin the physical model used to design the MPLC offline – a manually aligned MPLC will always\nsuffer reduced performance. We now include a new supplementary section SI §8 which shows\nsimulations of this effect – we see that even weak aberrations have a significant impact on MPLC\nperformance. In this case our self-configuring MPLC can automatically account for these aberra-\ntions, and so will surpass the performance achievable through manual alignment. We now make\nthispointinSI§8.\nIn our paper we have demonstrated self-configured MPLCs with fidelities of 87-95% and av-\nerage mode sorting cross-talk ranging from -17 to -19dB. The fidelity and cross-talk of an MPLC\ndepend upon a number of factors, such as the number of modes being transformed, the number of\nplanes, the distance between the planes, the number of pixels per plane, and the particular trans-\nformationenacted. Itisthereforenotstraightforwardtomakeadirectcomparisonwithourcurrent\nexperiments and previously published work – as the above parameters are typically different in\neach experiment. Nonetheless, we believe the experimentally achieved cross-talks and fidelities\nthat we present are, for our particular set of parameters, on-par with the state-of-the-art examples\nofwell-alignedMPLCsintheliterature–see,forexample,refs.[12–15].\nFurthermore, we emphasize that it is extremely challenging to manually align an MPLC, and\nour approach resolves issues that arise if manual alignment is not possible (as seems to have oc-\ncurred, for example, in ref. [16], where the researchers were limited to 2-plane MPLCs due to\nproblems manually implementing more planes). Indeed, we pursued this line of research in part\nbecause of the difficulties we experienced manually implementing a PLM-based MPLC – despite\nbeingalab.withsubstantialexperienceinthisarea(e.g.[9,12]). Wethinkthismayhavebeendue\ntotheunknownsurfacecurvatureofourPLM,sinceMEMSdevicestypicallyexhibitlowerflatness\nthanliquidcrystalSLMs[17]. Thismeantthatdesignsoptimisedusingtheconventionalwavefront\nmatching method, via a physical model with no knowledge of the PLM surface curvature, did not\nwell match the physical system. Evidently our self-configuring approach naturally accommodates\ntheseunknowns.\nNewly added SI §8 illustrates how sensitive MPLCs are to both unknown misalignments and\nunknownphaseaberrationsoftheplanes. Inthissectionwealsoshowtheperformanceofourself-\nconfiguring algorithm when initialised from a set of masks that have been pre-designed offline,\nbut are slightly misaligned. Interestingly, in this case we find that our self-configuring algorithm\nconverges more rapidly (roughly twice as fast) than if starting from flat phases, and finds a more\nefficient final solution than initialising blindly from flat phases. This suggests that first making\nbest efforts to align a pre-designed MPLC, followed by running our self-configuring approach is\na promising way to enhance MPLC performance. We now refer to these additional simulations in\nthe‘Optimisationtimescales’sectionofthemainpaper,wherewestate:\n“Combiningourautomatedapproachwiththesemethodscouldreducethenumberofmaskupdate\ncyclesC neededforthedesignprocesstoconverge–asexploredinsimulationsinSI§8.\"\nFurthermore, the ability to automatically optimise MPLCs in-situ, after only basic manual\nMPLCalignmentandirrespectiveofthenatureoftherequiredMPLCmodetransformation,offers\nseveral other advantages over pre-designed MPLCs. For example, beyond simple aberrations and\nmisalignments, our protocol allows for automatic alignment of MPLCs with unknown and poten-\ntially highly complex optical systems between the planes for which there is no physical model\navailable–somethingthatisnotpossiblewiththeconventionalwavefrontmatchingmethodwhich\nrelies on an accurate physical model. This represents a capability beyond any other free-space\nMPLC design approach as far as we are aware. We now make this point in the ‘Discussion and\nconclusions’sectionofthemainpaper,wherewehaveadded:\n“OurprotocolmakespossibletheimplementationofMPLCswithunknownandpotentiallyhighly\ncomplexopticalsystemsbetweentheplanesforwhichthereisnophysicalmodelavailable–some-\nthing that is not possible with the conventional wavefront matching method which relies on an ac-\ncuratemodeloftheopticalsystem.\"\nSimilarly, our algorithm can operate without knowledge of the light modes entering the MPLC,\nwith potential applications to reversing the scrambling of light due to unknown scattering me-\ndia[9]. Weareexcitedtoexplorethepossibilitiesopenedupbytheseadvancesinthefuture.\n2.2: Andwhatarethey(theresults)(ifatall)limitedby?\nResponse: Below we consider in more detail what limits the performance of our current experi-\nments,andsuggestpotentialfutureimprovementstomitigatethesefactors:\n(1) In our experiments, the number of TM samples (P) limits the highest spatial frequency of\nthe MPLC masks. Increasing P (up to a maximum of the total number of pixels on each plane)\nwould increase the resolution of the masks and thus improve the MPLC performance (see SI §2).\nHowever this also results in a longer optimisation time. Here strategies to reduce measurement\noverhead,asdiscussedinthe‘Optimisationtimescales’sectionofthemainpaperandinanswerto\nreviewerpoint1.6above,willbeexploredinthefuture.\n(2) The constrained field-of-view of our high-speed camera may also place a limit on the MPLC\nperformance. For example, in order to run the camera at a rate that could keep up with the PLM\nswitching rate, we had to reduce the field-of-view, meaning that after digital holographic process-\ning, the resolution of the final fields was relatively low (∼ 20 × 20 pixels). We think that using\na camera that can accommodate a larger field-of-view at these high frame rates will improve the\nmeasurementfidelityoftheoutputfields,andthusimprovethedesignprocess.\n(3) TM measurement noise will also contribute to inaccuracies in the mask update function. This\neffectcouldbereducedbyincreasingtheintensityofthelaserused(thereisamplescopeforthisas\nweusedalowpower1mWHeNelaserinourcurrentexperiments),andbyusingamoresensitive\ncamerawithalowernoisefloor.\n(4) Non-linear effects cannot be captured by a linear TM and so cannot be properly accounted for\nin our in-situ design process. The main source of non-linear effects is likely to be multiple reflec-\ntions between the MEMS mirrors and the coverglass in front of them. Such multiple reflections\nresult in a non-linear relationship between PLM mirror heights and the phase retardation of the\nreflected field. This nonlinearity is structural [18, 19], preserving the wavelength, and should not\nbe confused with optical nonlinearities (e.g. harmonic generation). This is only a problem on the\nmask from which the TM is being measured – we do not have to be concerned with such multiple\nreflectionsontheothermaskswhichareheldstaticandsotheeffectofmultiplereflectionscanstill\nbe captured by a linear matrix operator. We believe this effect is small, but would be further re-\nducedbyanti-reflectioncoatingonthePLMcoverglass. Furthermore,replacingTMmeasurement\nwith adaptive optimisation [4, 5], as mentioned in point 1.6 above, would circumvent the need to\nassumealinearmatrixandthusavoidtheeffectofsuchnon-linearities.\nWe have now include this information in a new supplementary section §12, which is also re-\nferred to in the ‘Arbitrary field reshaping and universal mode sorting’ section of the main text\nwherewestate:\n‘SeeSI§12forfurtherdiscussionontheexperimentalfactorslimitingourapproach,andpossibil-\nitiesforimprovementinfuture’\nOtherminormodifications:\n• We corrected Equation 10 by removing the denominator of the summand compared to our\noriginal submission. This constant denominator doesn’t cause any issues but was unneces-\nsary(asisconstant)andhadnotbeenimplementedinourexperiment.\n• We changed how we calculate the average cross-talk from the data in the cross-talk matrix.\nPreviously we first converted each off-diagonal value to dB and then averaged these dB\nvalues. We now average the off diagonal elements and subsequently convert this average\nvalue to dB. This second method slightly increased our quoted average cross-talk, but we\nthinkitisafairerrepresentationoftheaveragecross-talk.\nReferences\n1. López Pastor, V., Lundeen, J. & Marquardt, F. Arbitrary optical wave evolution with Fourier\ntransformsandphasemasks.OpticsExpress29,38441–38450(2021).\n2. Mastiani, B., Osnabrugge, G. & Vellekoop, I. M. Wavefront shaping for forward scattering.\nOpticsexpress30,37436–37445(2022).\n3. Mastiani, B., Cox, D. W. & Vellekoop, I. M. Practical considerations for high-fidelity wave-\nfrontshapingexperiments.JournalofPhysics:Photonics6,033003(2024).\n4. Vellekoop, I. M. & Mosk, A. Phase control algorithms for focusing light through turbid me-\ndia.Opticscommunications281,3071–3080(2008).\n5. Conkey, D. B., Brown, A. N., Caravaca-Aguirre, A. M. & Piestun, R. Genetic algorithm\noptimization for focusing through turbid media in noisy environments. Optics express 20,\n4840–4849(2012).\n6. Goel, S. et al. Simultaneously sorting overlapping quantum states of light. Physical Review\nLetters130,143602(2023).\n7. Fontaine, N. K. et al. Wafer scale fabrication of multi-plane light conversion devices in 49th\nEuropeanConferenceonOpticalCommunications(ECOC2023)2023(2023),1063–1066.\n8. Porte, X. et al. Direct (3+ 1) D laser writing of graded-index optical elements. Optica 8,\n1281–1287(2021).\n9. Kupianskyi, H., Horsley, S. A. & Phillips, D. B. All-optically untangling light propagation\nthroughmultimodefibers.Optica11,101–112(2024).\n10. Orr,G.etal.High-volumeproductiontestmethodologyandparametricsoftheTexasInstru-\nmentsPhaseLightModulator(PLM)inEmergingDigitalMicromirrorDeviceBasedSystems\nandApplicationsXVII 13383(2025),9–17.\n11. Lib, O., Shekel, R. & Bromberg, Y. Building and aligning a 10-plane light converter. arXiv\npreprintarXiv:2409.20039(2024).\n12. Kupianskyi, H., Horsley, S. A. & Phillips, D. B. High-dimensional spatial mode sorting and\nopticalcircuitdesignusingmulti-planelightconversion.APLPhotonics8(2023).\n13. Brandt, F., Hiekkamäki, M., Bouchard, F., Huber, M. & Fickler, R. High-dimensional quan-\ntumgatesusingfull-fieldspatialmodesofphotons.Optica7,98–107(2020).\n14. Lib,O.,Sulimany,K.&Bromberg,Y.Processingentangledphotonsinhighdimensionswith\naprogrammablelightconverter.PhysicalReviewApplied 18,014063(2022).\n15. Goel, S. et al. Inverse design of high-dimensional quantum optical circuits in a complex\nmedium.NaturePhysics20,232–239(2024).\n16. Martinez-Becerril,A.C.etal.Reconfigurableunitarytransformationsofopticalbeamarrays.\nOpticsExpress32,41111–41126(2024).\n17. Rocha, J. C. et al. Fast and light-efficient wavefront shaping with a MEMS phase-only light\nmodulator.OpticsExpress32,43300–43314(2024).\n18. Xia, F. et al. Nonlinear optical encoding enabled by recurrent linear scattering. Nature Pho-\ntonics,1–9(2024).\n19. Yildirim,M.,Dinc,N.U.,Oguz,I.,Psaltis,D.&Moser,C.Nonlinearprocessingwithlinear\noptics.NaturePhotonics,1–7(2024)."}, {"round": 2, "reviewer_comments": {"reviewer_1": "I can accept the manuscript as it has addressed the reviewers comments.", "reviewer_2": "The authors have satisfactorily addressed my comments. I strongly recommend that the work be published in Nature Communications."}, "author_response": ""}]

Decision letter