Multilayer nonlinear diffraction neural networks with programmable and fast ReLU activation function

Domain: physics
Published: 2025-11-24
Editorial outcome: major_revision

Abstract
Optical diffractive neural networks are emerging for improving speed and energy efficiency in machine learning. However, the challenges of nonlinear activation functions (e.g., latency issues, high power consumption, and cascading complexity) impede their performance and practical deployment. Here, we propose a programmable multilayer full-space nonlinear neural network operating in the microwave frequency band. Its nonlinear layers are constructed using programmable metasurfaces integrated with RF components, implementing a ReLU-like activation function. The nonlinear architecture achieves a nanosecond-scale delay (17.7 ns), representing orders of magnitude improvement in speed over photoelectric conversion-based nonlinearities. Moreover, the nonlinearity is characterized by exceedingly low thresholds and reconfigurable nonlinear activation functions. The system demonstrates remarkable classification capability in image classification and real-time human posture recognition tasks. Characterized by low latency, high speed, low power consumption, and flexible nonlinear activation, this architecture holds great promise for applications in security screening, medical rehabilitation, human-computer interaction, and numerous other fields.

Introduction
The surge in neural-network scale precipitates an exponential rise in energy expenditure and training cost, underscoring the need for more efficient alternatives1. Optical computing stands out as an effective substitute for electronic computation devices, particularly in the artificial intelligence applications, in which optical neural networks demonstrate marked superiority in speed and energy efficiency over their electronic counterparts2–13. Passive optical diffractive neural networks (DNNs) have faced challenges in programmability, whereas programmable metasurfaces provide real-time programmability14–16, which are crucial for applications like computational imaging17,18 and adaptive intelligent devices19. Recent advancements in DNNs in the microwave band include the programmable diffractive deep neural networks20, dynamic holography21, planar diffractive deep neural networks22, and surface plasmon polariton neural networks23,24. These studies highlight the microwave DNN’s inheritance of the key strengths in optical computing, including the high-speed processing, parallelism, and energy efficiency.

Considering that the normal diffraction process is linear, how to achieve efficient and high-speed nonlinear activation is a key in the research of DNNs. Hence, significant research efforts have been directed towards developing DNNs that are integrated with the nonlinear activation functions25–32. However, these nonlinearities face substantial practical limitations. For instance, the nonlinearity derived from semiconductor quantum dots (QDs)33 is inherently constrained by the conversion efficiency of QDs, which is characterized by low transmission and significant energy attenuation, impeding the cascading of multiple nonlinear layers. Optical nonlinear activation mechanisms, such as the photonic saturable absorbers29,34 and saturation of radio frequency amplifiers35, exhibit high thresholds and necessitate high-energy inputs to elicit the nonlinear responses. The conventional CMOS sensor-based photoelectric nonlinearities36,37 constrained by exposure duration exhibit millisecond (ms) or microsecond (μs) level latency, which impede the high-speed computational performance. Furthermore, the nonlinearities that rely on analog-to-digital converters (ADCs), digital-to-analog converters and microcontroller units (MCUs)23 demand complex peripheral devices, which significantly limit the system’s computation rate due to the processing constraints of digital circuits. Thus, these issues further impede the capability of DNNs to demonstrate systematic advantages over traditional digital computations in practical tasks.

To address these challenges, here we propose a multilayer nonlinear diffractive neural network (MN-DNN) with the programmable and fast Rectified Linear Unit (ReLU) activation function operating at 5.8 GHz. The network features multiple nonlinear layers composed of nonlinear metasurfaces integrated with RF detectors, amplifiers, and voltage adders, which exhibit the intensity-dependent nonlinear transmission coefficients, and hence can effectively act as the ReLU activation function. By selecting an optimized high-speed device, the engineered single-layer nonlinear metasurface achieves nanosecond-scale delays (17.7 ns), representing a substantial reduction compared to the millisecond (ms) and microsecond (µs) latency that is typically associated with conventional nonlinear optoelectronic conversion processes26,36,37. The ReLU thresholds can be adjusted through diverse bias voltages, liberating the nonlinear activation from depending on high-intensity illumination, and enhancing adaptability across varied intensity conditions. The gain of nonlinear metasurfaces is controllable via bias voltage, mitigating energy loss, and stacking of multiple nonlinear layers. We show that MN-DNN can enhance the image classification on Modified National Institute of Standards and Technology (MNIST) and Fashion-MNIST datasets, with the accuracy of 92.81% on MNIST and 78.8% on Fashion-MNIST, surpassing the linear networks by over 4% in both cases. It can also directly classify the human postures from electromagnetic waves in real time with the accuracy of 93.06%, surpassing the linear network’s accuracy of 81.85%. The advantages of this network in low latency and high accuracy make it ideal for applications in real-time sensing, motion detection, wireless communication, radar signal processing, and beyond.

Results
Principle of MN-DNN
To address the challenges of nonlinear activation, we propose an MN-DNN featuring low latency and adjustable nonlinearity for fast and accurate computations. The traditional DNNs require digital encoding of the input data, which increases the complexity and latency. Our architecture transcends the conventional encoding and enables the processing of both encoded input data and direct electromagnetic wave information, such as for real-time human posture classification shown in Fig. 1a. The direct processing significantly enhances the computational speed and substantially simplifies the hardware. The MN-DNN architecture comprises linear and nonlinear layers, and the latter employs the metasurfaces that incorporate RF amplifiers, detectors, and voltage adders to emulate the ReLU nonlinear activation function based on the power between input and output. Additionally, the implementation of diverse activation functions such as Hyperbolic Tangent (Tanh) and Leaky Rectified Linear Unit (Leaky-ReLU) is straightforward with the alternative active circuit configurations, as elaborated in the “Discussion” section.

In addition to real-time processing advantages, the MN-DNN architecture also exhibits strengths in its programmability and low latency. The nonlinearity is tunable via external bias voltage, enabling adjustments to the activation threshold and slope, as shown in Fig. 1b. This flexibility eliminates dependence on high power, broadening its applicability across various scenarios. Low latency is essential for swift information processing, but the nonlinearities in the current neural networks often introduce millisecond to microsecond delays owing to their reliance on optoelectronic conversion23,26,36,38, hindering the potential for computation at the speed of light. However, our architecture achieves a latency of 17.7 ns (Fig. 1c), marking a substantial reduction in the delay and underscoring its importance for real-time processing. Hence, MN-DNN boosts the information extraction capabilities, outperforming the linear networks with over 4.2% higher accuracy in both classifications of MNIST and Fashion-MNIST datasets, as depicted in Fig. 1d, e. The MN-DNN also enables real-time extraction of information from electromagnetic (EM) waves. As shown in Fig. 1f, it accurately classifies eight postures with the 93.06% accuracy, enhancing the computational efficiency and reducing the hardware complexity over the conventional methods. Hence, it can be used in human-computer interaction, intelligent surveillance, and industrial inspection.

Architecture of nonlinear metasurface unit
In MN-DNN, the nonlinear layer is implemented with a nonlinear metasurface. The nonlinear unit structure comprises four metal layers and three dielectric layers. The three dielectric layers, from top to bottom, are as follows: 0.5 mm-thick F4B (relative permittivity εr = 2.65), 0.2 mm-thick FR4 (relative permittivity εr = 4.4), and another 0.5 mm-thick F4B (relative permittivity εr = 2.65). The RF energy is coupled by the receiving antenna, passes through the active circuit, and is radiated by the transmitting patch antenna into the air, as illustrated in Fig. 2a.

The front-side passive structure comprises an octagonal receiving patch antenna and a T-junction unequal power divider, as shown in Fig. 2b. Optimized for operations at 5.8 GHz, the patch antenna exhibits a side length dimension of D = 7.57 mm to achieve proper resonance characteristics. For impedance matching, the octagonal patch antenna is slotted with a length L1 = 4.91 mm and a width W1 = 1 mm. The microstrip line width W2 is 1 mm. The detection arm of the T-junction features a two-stage impedance transformation, refined by the corner chamfering to ensure a smoother impedance transition. The first branch has a width W3 = 0.5 mm with length L3 = 1.746 mm, while the second branch has a width W4 = 0.254 mm with length L4 = 1 mm. The unit’s back-side features an octagonal transmitting patch antenna identical in size to the receiving antenna, as illustrated in Fig. 2b. The operational diagram of the nonlinear metasurface unit is shown in Fig. 2c. The spatial RF energy, post-coupling by the receiving antenna and microstrip line transmission, is split by the T-junction power divider. The bulk of energy is amplified by the RF amplifier and proceeds to the transmitting antenna, while a fraction of the energy is diverted to a high-speed RF detector and is converted into direct current (DC) voltage VOUT. The combined voltage of VOUT and external bias voltage VBIAS is summed by a voltage adder: VCTRL = VOUT + VBIAS. This control voltage is used to modulate the amplifier. Details of the nonlinear metasurface unit structure are presented in Supplementary Note 1.

The RF detector’s output voltage VOUT is positively correlated with the input RF energy, as illustrated in Fig. 2d. The gain of RF amplifier is modulated jointly by DC voltages VCC and VCTRL, as shown in Fig. 2e. The effect of DC voltages VCC on the RF amplifier’s gain is presented in Fig. 2f. We note that the gain falls below −15 dB when the voltage is less than 0.5 V, effectively muting the amplifier with negligible energy transfer. Above the threshold, increasing VCC will shift the gain into the amplification zone and reach a stable gain. Fig. 2g indicates that low VCTRL values result in a transmission coefficient below −10 dB, halting the transmission. Above 1.2 V, VCTRL initiates the signal amplification, maintaining a stable gain despite further increases. With a high external bias voltage VBIAS (e.g., VBIAS = 1.5 V), the amplifier’s VCTRL surpasses the amplification threshold, independent of the detector’s output voltage VOUT. As shown in Fig. 2h, In the linear amplification regime, the output power of the metasurface unit is linearly proportional to its input, acting as a linear amplitude modulator with VCC governing the amplitude of the transmission coefficient.

With low VBIAS, the unit is operated nonlinearly, and its transmission coefficient changes with incident power. By increasing the incident power, the detector’s VOUT and the amplifier’s control voltage VCTRL rise linearly. The amplifier gain rises from below −10 dB to a stable level, and the metasurface unit’s transmission coefficient increases from a low value to a stable state. Higher VBIAS leads to an earlier transition into the linear amplification, while lower VBIAS requires more input power to reach the turning point of transmission coefficient, as shown in Fig. 2i. The transmission coefficient S21 escalates with VCC magnitude under constant input power and VBIAS, as illustrated in Fig. 2j. We note that the detector’s VOUT and the amplifier’s VCTRL are low when the incident power on the metasurface is below 1 mW, leading to amplifier cutoff and zero output from the metasurface unit. Beyond 1 mW, the voltages will be increased, activating the amplifier’s linear amplification and causing the metasurface unit’s output power to rise linearly with the input power, which emulates the ReLU function in neural networks, as demonstrated in Fig. 2k. In this scenario, VBIAS is 0.5 V and VCC is 5 V. Figure 2l indicates that a higher VBIAS reduces the nonlinear activation function’s threshold, enabling variable nonlinear activation thresholds by modulating the bias voltage VBIAS. Figure 2m shows that increasing VCC steepens the slope of the nonlinear activation function, resulting in elevated amplifier gain and transmission coefficients for the metasurface units. Based on the measured data (details of the nonlinear metasurface tests are provided in Supplementary Note 2), we construct scatter plots and employ curve fitting to derive the mathematical expression of the activation function, which exhibits a ReLU-like form:

y = b × max(0, x − a) (1)

where x represents the input power, y denotes the output power, a is the threshold parameter, and b is the slope parameter.

This nonlinear activation function offers 17.7 ns latency in the measurements (see Supplementary Note 3 for details). Each nonlinear unit exhibits a total power dissipation of approximately 429 mW, which consists of 150 mW for the RF amplifier, 53 mW for the RF detector, and 224 mW for the voltage adder. Supplementary Note 4 and Supplementary Table S2 compare the performance of diverse nonlinear implementation methods in DNNs, focusing on metrics such as time delay, threshold, multi-layer stacking, and programmability.

Handwritten digits classification by MN-DNN
The primary objective is the classification of MNIST dataset, which involves identifying ten classes of handwritten digits ranging from 0 to 9, as shown in Fig. 3a. The original 28×28 pixel images are down sampled to 14×14 pixels to match the dimension of the input-layer metasurface via bilinear interpolation, and subsequently binarized to values of 0 or 1. Further details on image processing for recognition are outlined in Supplementary Note 5, with the effect of image preprocessing on recognition accuracy analysis provided in Supplementary Note 6. Intensity modulation is achieved by mapping the input image pixels onto the first-layer metasurface. The background is opaque to the EM waves, with the digits being transparent (transmission coefficients of 0 dB for pixel 0 and 10 dB for pixel 1), as illustrated in Fig. 3b. Consequently, the EM wave conveys the information of the input image as it passes through the first layer.

As shown in Fig. 3d, we independently train linear and nonlinear neural networks using the full MNIST dataset (60,000 for training and 10,000 for testing) to evaluate their image classification performance. The linear network comprises three linear phase-modulation layers spaced 20 cm apart, each composed of a 22×22 array of 19×19 mm2 units. The phase of each unit serves as a trainable parameter, adjustable in the range from 0 to 2π (see details in Supplementary Note 7). The nonlinear network features an alternating linear-nonlinear structure with three linear layers interspersed with three nonlinear metasurface layers, spaced at 10 cm intervals. Each nonlinear layer contains two independent trainable parameters: the threshold and slope of the ReLU activation function, with layer-specific values (see Supplementary Note 8 for the RF component math modeling). The nonlinear network consists of 1458 trainable parameters, with 484 parameters per linear layer and 2 parameters (threshold and slope) per nonlinear layer, yielding a total of 484 × 3 + 2 × 3 = 1458, and all parameters are trained together. In comparison, the standard fully connected networks for the MNIST classification typically require tens to hundreds of thousands of parameters, whereas our model achieves an order-of-magnitude reduction with only ~1500 parameters. The linear and nonlinear networks remain fully independent throughout their design and training phases, without shared parameters or cross-influence. The loss function is a weighted sum of the Mean Squared Error and SoftMax Cross-Entropy, with coefficients 0.4 and 0.6, respectively. The gradient descent optimization is performed using the Adam algorithm. The mathematical model of DNNs is detailed in Supplementary Note 9. The optimization procedure explicitly accounts for the field distribution of 5.8 GHz horn antenna on the input plane under the test conditions (Supplementary Note 10), to reduce the discrepancies between the simulations and measurements. The training outcomes indicate that the nonlinear network achieves a recognition accuracy of 92.81%, which is 4.18% higher than the 88.63% accuracy of the linear network. It is important to note that this improvement does not result from the increase of trainable parameters, as each nonlinear layer only adds two trainable parameters (threshold and slope), and three nonlinear layers contribute a total of six additional parameters. For comparison, a linear network with the same number of parameters achieves at most 89% accuracy, further confirming the performance gain from the nonlinear activation functions (see Supplementary Note 11). The output fields of both networks are reduced to two dimensions via t-SNE, as shown in Fig. 3e. The clustering of the linear network shows a looser intra-cluster structure with overlapping data points across categories, whereas the output of the nonlinear network displays higher local density within clusters and clearer separation between categories.

Based on the simulation results, nonlinear and linear network samples are fabricated, in which the metasurfaces are mounted on acrylic plates with plastic screws and aligned into the grooves of aluminum alloy brackets. Absorptive materials are applied to the plates and brackets to reduce the edge diffraction of EM waves and to minimize their influence on the EM fields, see Supplementary Note 12 for details. The nonlinear prototype includes an input layer, three linear layers and three nonlinear layers of metasurfaces. The input-layer metasurface functions as a configurable linear amplitude modulator in a 0–10 dB range, which is implemented by a nonlinear metasurface in linear amplification mode, controlled by a bias voltage. It consists of 14×14 units, where the transmission coefficient of each unit matches the input image pixel values, modulated by an MCU-regulated bias voltage. The activation function parameters of nonlinear metasurfaces are modulated by the external DC voltage. The linear prototype comprises an input-layer metasurface and three linear phase-modulating metasurfaces, configured analogously to the nonlinear network. Both linear and nonlinear prototypes are excited by a 5.8 GHz horn antenna.

The input-layer encoding was performed through intensity modulation. As depicted in Fig. 3c, a two-dimensional near-field scanning platform recorded the output field distributions. To comprehensively evaluate the network performance, we systematically assessed the recognition accuracy of both networks using 5000 randomly selected images. In the experimental setup, a rapid detection device was used to replace the conventional two-dimensional near-field scanning stage (see Supplementary Note 13), enabling efficient energy measurements across ten specific regions of the output field for each image. This approach significantly enhanced the detection throughput, allowing rapid acquisition of classification results for large-scale image datasets. The MNIST rapid classification experiments conducted with this device are presented in Supplementary Video 1. The experimental results demonstrate that the nonlinear network achieves a classification accuracy of 92.6% on the 5000 MNIST images (compared to a simulated accuracy of 92.81%), exhibiting a clear advantage over the linear network (88.5% accuracy), as shown in Fig. 3f. To further analyze the networks’ behavior, we performed output field distribution scans on a subset of images. For simple images with distinct features such as the handwritten digit “7” (Fig. 3g), both linear and nonlinear networks achieve accurate recognition, with their output energies precisely concentrated on the target category’s region. However, when processing complex images with easily confusable features, the nonlinear network shows a pronounced advantage. As seen in Fig. 3h, for a “4” sample prone to confusion with “9”, the nonlinear network’s output energy is correctly localized to the ‘4’ region, whereas the linear network misclassifies it, with its peak energy erroneously appearing in the “9” region, although the “4” region has the second-highest energy. This phenomenon remains consistent across other complex image classifications, demonstrating the nonlinear network’s superior performance in handling complex images. Further comparisons are presented in Supplementary Note 14. A detailed comparison between simulated and experimental results is given in Supplementary Note 15. The total power of MN-DNN is about 252.352 W (see Supplementary Note 16 for details).

Fashion-MNIST classification by MN-DNN
Next, we compare the classification performance of linear and nonlinear networks on the complete Fashion-MNIST dataset, which consist of 60,000 training samples and 10,000 test samples. This dataset comprises ten apparel categories: T-shirts, trousers, pullovers, dresses, coats, sandals, shirts, sneakers, bags, and ankle boots, as illustrated in Fig. 4a. The 28×28 pixel images are resized to 14×14 pixels to match the input layer metasurface using bilinear interpolation and then binarized. Intensity modulation is applied by mapping the pixel values onto the first-layer metasurface. To assess the image classification capabilities of linear and nonlinear neural networks, we optimize both structures to map the output field distributions of ten classes to ten distinct regions, as shown in Fig. 4b. The network architectures for Fashion-MNIST recognition mirror to the architectures employed in MNIST, and both linear and nonlinear networks undergo independent optimization processes without any parameter sharing. The nonlinear diffraction network achieves 78.8% accuracy on Fashion-MNIST, surpassing the linear network (74.43%) by 4.37 percentage points. Similar to the previous case, this improvement is not due to the increased parameter count, as each nonlinear layer only introduces two trainable parameters (the threshold and slope) with a total of six additional parameters, which are negligible compared to the model’s 1458 parameters. Under the same parameter conditions, the linear network achieves 75.08% accuracy (Supplementary Note 11), further demonstrating the performance gain from the nonlinear architecture. The accuracy comparison with other networks is given in Supplementary Note 17. Based on the simulations, we fabricate the nonlinear and linear prototypes. During experimental evaluation, a set of 5000 randomly sampled Fashion-MNIST test images is intensity-encoded and processed through both nonlinear and linear network configurations. The recorded recognition accuracy reaches 77.4% and 72.5% for the nonlinear and linear networks, respectively. The resulting output field distributions of linear and nonlinear samples from the horn antenna irradiation are shown in Fig. 4c, d. For simple and distinct images like the ankle boot in Fig. 4c, both linear and nonlinear networks achieve accurate classification, with the output energy correctly focused on the target region. However, when dealing with complex images characterized by similar and confusable features, the nonlinear network demonstrates a clear advantage. As illustrated in Fig. 4d, the nonlinear network correctly classifies the sneaker image (often confused with ankle boots), with the output energy concentrated specifically in the sneaker region. In contrast, the linear network misclassifies this input, displaying the peak energy in the wrong ankle boot region despite the secondary activation in the correct sneaker category. This consistent pattern across challenging images (see Supplementary Note 18 for more results) confirms the nonlinear networks’ superior performance in recognizing the complex images.

In addition to the experiments, we conduct a series of simulation analyses to determine the impact of the number, placement, and type of activation functions in nonlinear layers on the recognition accuracy. We assess different network configurations with varying numbers of nonlinear layers: a three-layer purely linear network; a network with an additional nonlinear layer between the first and the second linear layers; a network with two nonlinear layers interspersed among the linear layers; and a network with three nonlinear layers evenly distributed among the linear layers. The results show that incorporating nonlinear layers can significantly enhance the recognition accuracy over the purely linear network, with the accuracy increasing as the number of nonlinear layers grows, as illustrated in Fig. 4e. In the three-linear-layer framework, we introduce nonlinear layers at three different locations: between the first and the second linear layers (front), between the second and the third linear layers (middle), and between the third linear layer and the output plane (rear). The results illustrate that the recognition accuracy increases with the addition of nonlinear layers at all positions, with the largest improvements at the front and rear, as depicted in Fig. 4f. In a three-linear-layer structure, we introduce a nonlinear layer preceding the output plane to test the ReLU, SoftMax, and Tanh activation functions. Each function boosts the accuracy significantly, with ReLU showing the greatest enhancement, as depicted in Fig. 4g. For more details, refer to Supplementary Note 19.

Static posture recognition by MN-DNN
We develop a posture recognition system using MN-DNN, as shown in Fig. 5a. A test person stands in front of MN-DNN and performs various postures, which affect the EM-wave focusing on the output plane. Our classification targets include eight distinct human body postures: stand, arms down, cheer, hands up, greet, scratch head, left arm, and sideway. Firstly, we collect a dataset of the postures. The system employs a standard horn antenna operating at 5.8 GHz to illuminate the test person. Posture information is extracted using an eight-element patch antenna array with 3-2-3 arrangement (5.8 GHz resonant frequency) that samples the scattered field distributions from the human body. Signals from these antennas are sequentially captured by Vector Network Analyzer (VNA), managed by a computer that oversees an MCU to control a single-pole eight-throw RF switch (refer to Supplementary Note 20). Four volunteers are designated as trainers and perform extensive data acquisition for eight postures under various conditions, as depicted in Supplementary Fig. S42. These include different rotation angles of human body relative to the antenna array (ranging from −30° to 30° in 10° increments) and varied distances between the human body and antenna array (from 20 cm to 50 cm in 10 cm increments), as presented in Supplementary Note 21. The input layer of our system features 8 active units with a transmission coefficient of 10 dB, arranged in a 3-2-3 array configuration, aligning with the antenna array’s positioning during the dataset acquisition. These units are tasked with spatially sampling the scattered EM waves emitted by the human body. The remaining input layer units are inactive, blocking the transmission of corresponding EM waves, as illustrated in Fig. 5a. The strategic selection of 8 sampling points in the input layer is crucial to attain high recognition accuracy in the limited dataset acquisition timeframe (see Supplementary Note 22 for more details). A total of 850 data samples are collected for each posture, resulting in 6800 samples across eight postures, with corresponding labels ranging from 0 to 7. Among these, 70.5% are allocated to the training set, 23.5% to the test set, and 6% to the validation set (see Supplementary Table S9 for more details).

We further demonstrate a low-latency and high-fidelity posture recognition task based on the proposed MN-DNN architecture. In traditional methods, the input data streams are processed in the digital domain, which has limitation in processing speed and requires complex front-end analog-to-digital conversion modules. In the proposed MN-DNN, the scattered field undergoes neural network computation during its propagation through the network, thereby overcoming these limitations. The MN-DNN comprises three linear layers and three nonlinear layers, as previously described and depicted in Fig. 5b. The output field distributions for various postures are mapped to eight distinct regions on the output plane. Simulation results show that the nonlinear network can quickly achieve a recognition accuracy of 93.06%, which substantially surpasses the linear network’s 81.85%, as detailed in Supplementary Note 23.

A trainer is invited to perform the test, as depicted in Fig. 5c. The EM intensity across eight specific regions on the output plane corresponds to the classification of eight distinct postures. In each region, a patch antenna coupled with an RF detector is utilized to rapidly measure the energy of the area, converting it linearly into DC voltages (V1–V8). The outputs from these detectors are connected to the ADC ports of the MCU for voltage detection. MCU compares the voltage magnitudes and controls the color of LEDs based on the channel with the maximum voltage. Each posture classification is associated with a unique LED color: “stand”—white, “arms down”—red, “cheer”—yellow, “hands up”—green, “greet”—blue, “scratch head”—light blue, “left arm”—purple, and “sideways”—orange. Additionally, MCU conveys the voltage values of the eight channels to a personal computer for real-time display (see Fig. 5d). The underlying principles of energy detection apparatus on the output plane are elaborated in Supplementary Note 24.

The tester performs each of the 8 postures 50 times, and the output voltages from eight RF detectors are logged to measure the intensity across eight distinct regions. Classification is determined by the posture corresponding to the detector with the highest voltage, as shown in Fig. 5e, where the mean output voltage of the detectors is represented by bar heights and their variances by error variances by error bar lengths. Consistent with expectations, the regions with the maximum energy on the output plane align with the executed postures. For example, during the “scratch head” posture, Detector 6 records an output voltage nearing 1.2 V, while others remain below 0.5 V, indicating the highest voltage and thus classifying the posture as “scratch head” in accordance with Fig. 5d. More experimental pictures are presented in Supplementary Note 25. To evaluate the system’s robustness, we conduct experiments with four additional participants to exhibit diverse body shapes and clothing in complex backgrounds, none of whom had participated in the initial data collection. The results show an average recognition accuracy of 87%. Future improvements could be focused on expanding the data collection scope by incorporating more sampling points and recruiting additional participants with varied body shapes, clothing styles, locations, and environmental conditions, which would enhance the system’s adaptability and robustness across diverse scenarios (see Supplementary Note 26 for detailed analysis).

Dynamic posture recognition by MN-DNN
MN-DNN can effectively recognize both static and dynamic postures. The tester performs a sequence of postures, holding each for approximately 4 s before transitioning to the next. The sequence encompasses stand, arms down, cheer, hands up, greet, scratch head, left arm, and sideway, as shown in Fig. 6a. The simulated field distributions corresponding to these postures on the output plane are presented in Fig. 6b, with the field’s focus moving sequentially through eight regions in chronological order. The output voltages from detectors are interfaced with MCU, which are then transmitted to a computer in real time via serial communication. Figure 6c presents the temporal voltage waveforms. During the interval from 0 to 4.3 s, the posture is “stand”, with Detector 1 exhibiting an output voltage of about 1.2 V, while other detectors show less than 0.55 V, resulting in its classification as “stand”. From 4.3 to 8.2 s, as the posture shifts to arms down, the voltage in Detector 1 drops significantly, and the voltage in Detector 2 increases to approximately 0.9 V, peaking and classifying the posture as “arms down”. This pattern holds for the subsequent postures, ending at the 32-s mark. Throughout each posture’s duration, the relevant detector’s output voltage dominates, allowing for accurate classification of dynamic movements despite minor outputs from other detectors. The complete tests are given in Supplementary Video 2. The total system latency is 2.48 μs, comprising both propagation delay through MN-DNN and processing delay incurred by the output detection. This represents an at least three-order-of-magnitude reduction in latency compared to the conventional camera systems coupled with digital neural network post-processing, which are typically operated with millisecond-level latency. For posture recognition tasks, real-time performance typically requires a time delay on the order of milliseconds39–41. The proposed system reduces the latency by at least three orders of magnitude, enabling efficient real-time motion recognition. By implementing a higher-speed output detection scheme with high-speed ADC modules and high-performance FPGA, the total system latency could be further reduced to 66.5–68.5 ns. See Supplementary Note 27 for details.

Methods
Time delay measurement of a nonlinear metasurface
The time-delay measurement system employs a 5.8 GHz signal generator to produce a continuous radio-frequency (RF) signal. This signal undergoes amplitude-shift keying modulation through a square wave (50% duty cycle) received from a waveform generator via an external trigger interface, thereby generating a modulated RF signal with a square-wave envelope. The modulated signal is then radiated by a transmitting antenna to excite the metasurface. To minimize the edge diffraction effects, we implement a coding strategy that activates only the central unit while deactivating the surrounding edge units, which are also covered with microwave-absorbing material. For real-time monitoring of the incident signal on the metasurface, a receiving antenna (Antenna 1) is positioned equidistant from the transmitting antenna and the central unit of metasurface. The RF signal captured by Antenna 1 is transmitted via a coaxial cable to Channel 1 of an oscilloscope. Simultaneously, a second receiving antenna (Antenna 2), placed on the opposite side of the metasurface, detects the transmitted wave and relays the signal through a coaxial cable of identical length to Channel 2 of the oscilloscope. By analyzing the temporal difference between the signal envelopes of two channels, the transmission delay introduced by the single-layer nonlinear metasurface is precisely quantified. For more details, see Supplementary Note 3.

Data collection for posture recognition
Trainers 1 to 4 conduct comprehensive data collection across eight human postures: stand, arms down, cheer, hands up, greet, scratch head, left arm, and sideway. Positioning themselves between a horn and an antenna array, they perform these postures facing the antenna array. The array captures scattered EM waves, which are then transmitted to a VNA for computer recording. To enhance dataset robustness, data collection encompasses a spectrum of scenarios, including body rotations (from −30° to 30° in 10° increments) relative to the antenna array’s normal direction and distances (from 20 cm to 50 cm in 10 cm increments) to the array. A total of 850 datasets are collected for each posture, resulting in an aggregate of 6800 across all postures, which are labeled from 0 to 7. These datasets are randomized and allocated into training (70.5%), testing (23.5%), and validation (6%) sets. Additionally, to further improve dataset robustness, we increase sampling points from 8 to 21. Eight participants with diverse body types and clothing materials perform eight postures repeatedly under both controlled and interference-rich conditions (5.8 GHz WiFi with multipath reflections). As shown in Supplementary Note 26, this enhanced dataset demonstrates significantly improved robustness in simulations.

Analysis of the count and position of nonlinear layers
(1) Number of nonlinear layers. We analyze the influence of nonlinear layer count on recognition accuracy using the MNIST and Fashion-MNIST datasets. The primary focus is on MNIST, where four distinct neural network configurations were simulated: a purely linear network with three layers, and three nonlinear networks with additional one, two, and three nonlinear layers, respectively. The architectures and simulation outcomes are detailed in Supplementary Fig. S35a–d, with the scatter plots illustrating the output fields post t-SNE dimensionality reduction. These plots reveal that increased nonlinearity enhances output field clustering, with intra-class samples clustering more tightly and inter-class boundaries becoming more distinct. Supplementary Fig. S35e corroborates that recognition accuracy escalates with an increment in nonlinear layers. Supplementary Fig. S36 delineates the Fashion-MNIST recognition outcomes across the four network categories, with accuracy escalating from 74.43 to 78.8%, underscoring a positive correlation between nonlinear layer count and recognition precision. Collectively, the findings from both datasets suggest that augmenting nonlinear layers positively influences image recognition accuracy.

(2) Position of nonlinear layers. We investigate the effect of nonlinear layer positioning on the recognition accuracy using MNIST and Fashion-MNIST datasets. The analysis focuses on four neural networks, each with three linear layers and a single nonlinear layer inserted at different points: inter-first-second, inter-second-third, and post-third linear layers. The detailed architectures and corresponding simulation results on the MNIST dataset are provided in Supplementary Fig. S37a–d. Results demonstrate that the introduction of nonlinear layers at any position improves recognition accuracy, with the highest gains observed when the nonlinear layer is near the output layer. This indicates that the performance is sensitive to the placement of the nonlinear components. Supplementary Fig. S38 shows the classification performance of these network configurations on the Fashion-MNIST dataset, confirming the MNIST dataset trend with the most significant accuracy improvements when the nonlinear layers are positioned near the output layer and the least improvement when placed in the middle.

Peer review rounds
[{"round": 1, "reviewer_comments": {"reviewer_1": "The authors have proposed and demonstrated an RF diffractive neural network that features linear and nonlinear layers using RF components. They show how to configure the nonlinearity using electrical bias and gain. Different pattern and posture recognition tasks are demonstrated. It is an interesting implementation, the authors have performed several tests, and the paper is well presented. However, there are several comments that should be addressed before it can be considered for publication:\n1- How did the authors choose the frequency of 50 MHz? This seems to make the scaling challenging. The antenna size decreases as frequency goes higher. Also, in the abstract they call the system “microwave” which is not accurate unless it is designed for a much higher frequency.\n2- It takes a few seconds for the dynamic posture recognition result to settle. As a fundamental question, if one uses a camera and a digital neural network (on CPU, GPU, …), how long would that take and how does that compare with the proposed architecture? I understand the benefit of not needing RF to digital conversion, but this is important at a system level.\n3- The adder that is used in the experimental setup uses TLV9001 opamp with a gain bandwidth product of 1 MHz (1us time constant). How can the system achieve a latency of 4.5ns as the adder seems to be involved in the forward path?\n4- The images are binarized. What would happen if the image resolution were higher than just one bit? How would the systems operate? This is an important point as a more realistic application requires higher resolution.\n5- The linear and nonlinear networks are trained separately. Is this optimal? Could the authors train the whole network together?\n6- How much is the total power consumption of the system? How does it compare with other approaches?\n7- Can the authors explain the digital training algorithm a bit more? Have they modeled the behavior of the RF components in the training software?\n8- In Fig. 2 the word “backward” is a bit confusing. One may think that signal propagates backward, too. Transmit and receive antenna might be a better choice.\n9- Authors mention (line 118) that the nonlinearity in neural networks is in milliseconds and microseconds range. What kind of neural network are they referring to? If it is optical, then much faster nonlinearities have been demonstrated. Please clarify.\n10- Line 174, what does the word “atom” refer too? Seems to be a typo.", "reviewer_2": "In \"Multilayer nonlinear diffraction neural networks with programmable and fast ReLU activation function\", the authors propose a microwave-domain multilayer nonlinear diffractive neural network (MN-DNN) that integrates programmable metasurfaces with RF detectors, amplifiers, and voltage adders to realize a fast, programmable ReLU-like activation with nanosecond latency. Each metasurface layer performs linear transformations, while custom electronic circuits implement the nonlinear activations by detecting, filtering, and re-amplifying signals. This hybrid design enables fast and low-power processing with tunable thresholds and gain, supporting other activations like Tanh and Softmax through simple circuit modifications. The system achieves 92.83% accuracy on MNIST, outperforms linear models by over 4% on both MNIST and Fashion-MNIST, and classifies human postures with 93.06% accuracy. Its compact, reconfigurable architecture offers low latency and high energy efficiency, addressing key limitations of prior diffractive systems and highlighting the potential of combining optics-inspired principles with active electronics for fast, efficient neural processing.\nThe authors have presented their work in a clear and articulate manner. However, the manuscript would benefit from further clarification on a few key points, as well as a more thorough validation of the proposed approach through additional testing. These improvements would significantly enhance the overall strength and impact of the article. I have pointed out these clarifications item by item for the authors' consideration below.\n1- On page 3, line 81, you state that the MN-DNN model demonstrates an enhancement in image classification performance on the MNIST and Fashion-MNIST datasets, achieving an accuracy of 92.83% on MNIST. However, the classification accuracy for the Fashion-MNIST dataset is not mentioned until much later, as far as I can see. To improve clarity and provide a more comprehensive introduction, it would be beneficial to also include the classification accuracy for Fashion-MNIST early on in the paper, to give the readers a quantitative anchor and also to be able compare the result with literature.\n2- It is obvious that the nonlinear results in Fig 3 are surpassing the linear results, as this is already expected from this type of a classification task. However, all the examples shown here are in Fig 3f belong to cases where the digits are misclassified by the linear network. For comparison purposes, I think it would be more valuable to demonstrate also one example where the linear network can also perform well. Otherwise, it seems like the linear network cannot classify any of the images, which is clearly incorrect since the classification accuracy reported is 88% in Fig 3g. The same can also be said about the fashion MNIST classification in Fig 4. Here, it seems like the nonlinear and linear networks have returned different results for all inputs shown, which clearly cannot always be the case with over 70% accuracy.\n3- From a similar perspective, the nonlinear network has significantly more layers and potentially more trainable parameters, which depends on how nonlinearity was trained. Could the improved performance be partially attributed to the increased network depth rather than solely the nonlinearity? A comparison with a 6-layer linear network, or one that has an equal number of trainable parameters would be more informative. The same problem also exists for the analysis in Supplementary Figure S12 and S13. Showing that additional nonlinear layers increases accuracy does not constitute a novel finding on its own, as this is already well-known and expected.\n4- In the dataset-based recognition tasks for both MNIST and Fashion-MNIST, as outlined in Supplementary Note 4, you describe a process in which the original 28×28 pixel grayscale images are first downsampled to 14×14 pixels using interpolation. Following this, a binarization is applied, where pixel values exceeding a threshold of 0.15 are set to 1. While this downsampling and binarization process is a standard approach in some image processing tasks, it is important to note that the binarization step can have a significant impact on dataset difficulty and model performance. By reducing pixel precision, one inevitably loses valuable information, which could directly contribute to a decrease in recognition accuracy. It would be beneficial to discuss the potential effects of this binarization on the presented results, as understanding its impact is crucial for readers evaluating the robustness of this approach.\n5- The binarization process inevitably leads to a decrease in performance, even when employing fully digital neural networks. Specifically, when this dataset is used for classification in digital neural networks, the accuracy values observed in the literature are typically lower. Notably, in the case of Fashion-MNIST, you observed a maximum accuracy of only 77% when utilizing ReLU activation, which is substantially lower than the results achieved in the original dataset for digital classification. Can this discrepancy be attributed to the downsampling and binarization techniques applied? How can one explain the significant drop in efficiency compared to typical digital models that achieve over 90% on the same task?\nIn other words, can the observed decrease in accuracy be more a result of the dataset transformation, rather than a limitation of the MN-DNN model itself? To further investigate this, I recommend training a traditional digital neural network (such as a CNN or FCNN) with an equivalent number of trainable parameters to those in MN-DNN, using the downsampled and binarized MNIST and Fashion-MNIST datasets. By comparing the performance of this digital network against the results from MN-DNN, the supplementary material could offer valuable insights into the specific impact of the dataset preprocessing on the performance and highlight any differences between MN-DNN and standard digital neural networks.\n6- For dynamic gesture recognition, I do not agree with the “real-time” claim of the authors. For a system to be called real time, it needs to be much faster compared to the response time of humans. Typical systems that are quoted as “real-time” operate at over 30Hz refresh rates, which is much faster than the response of this system. Moreover, the response time of the presented system currently can only be inferred from Fig 6c, as no formal analysis is presented. Note that this is different from the 4.5ns delay for the metasurface layer shown in Fig 1. I suggest the authors actually quantify this response time, discuss factors contributing to it and comment on how the system can be made faster.\n7- While the threshold for activation is low, what is the actual power consumption of the active components (amplifiers, detectors, adders) per unit cell, and for the entire system? How does this compare to other DNN approaches when considering the entire network?\n8- The manuscript does not provide a detailed discussion of the number of trainable parameters, which is a crucial aspect for understanding the model's complexity and scalability. It would be beneficial for the authors to explicitly mention the total number of trainable parameters and offer a comparison with those typically found in conventional neural networks. Furthermore, the number of trainable parameters associated with both the linear and non-linear layers should be clearly specified to provide a more comprehensive understanding of the model's architecture and its computational demands. A more thorough discussion in this regard would improve the transparency of the model's design and facilitate a clearer comparison with existing approaches in the literature.\n9- The training and test process of the MN-DNN model on both the MNIST and Fashion-MNIST datasets lacks clarity regarding the sample sizes used. On page 11, line 269 of the main text, it is mentioned that 200 samples were randomly selected for the MNIST dataset. However, considering that MNIST consists of 70,000 samples, the selection of only 200 samples would be insufficient for a robust evaluation. A larger test sample size would provide more reliable and representative accuracy metrics. Was there a specific reason why such a small sample set was used? In this case, I would recommend that the authors increase the test sample size and re-evaluate the accuracy values accordingly. Additionally, the test sample size for the Fashion-MNIST dataset is not specified, which should be addressed to ensure the comparability and reliability of the results. Clear documentation of the sample sizes for both datasets is essential to assess the validity of the reported findings.\n10- Some information should be clarified regarding the robustness of posture recognition. The authors collected data from 4 volunteers under controlled settings (angles, distances, etc.). How robust is the system to variations in various changes in data such as clothing, body size/shape, different individuals not in the training set, or other complex background environments? The use of only 8 spatial sampling points seems sparse; is it not possible to use more sampling points? Also, how sensitive is the system performance to mispositioning of the person relative to these points?", "reviewer_3": "I co-reviewed this manuscript with one of the reviewers who provided the listed reports. This is part of the Nature Communications initiative to facilitate training in peer review and to provide appropriate recognition for Early Career Researchers who co-review manuscripts.", "reviewer_4": "The paper by Ning et al presents a highly novel contribution to the field of diffractive neural networks (DNNs) by introducing a multilayer nonlinear diffractive neural network (MN-DNN) operating in the microwave band. This system leverages programmable metasurfaces integrated with RF amplifiers, detectors, and voltage adders to implement a tunable ReLU-like nonlinear activation function with an impressive 4.5 ns latency, significantly faster than the millisecond/microsecond latencies of conventional optoelectronic methods. The MN-DNN's ability to adjust activation thresholds and slopes via bias voltages enables real-time reconfiguration, addressing the rigidity of prior optical DNNs. Additionally, its low-power nonlinear processing and direct electromagnetic (EM) wave processing for tasks like human posture classification set it apart from existing approaches, which often rely on high-energy inputs or digital encoding. The research represents an impactful contribution to both research and practical deployment. I am happy to recommend its publication in Nature Communcations provided the authors could address the following points.\n1. In the introduction, the authors described the advantages of their DNNs in comparison to other approaches with nonlinear activation, including the short latency and low threshold. The authors also made comparison of the accuracy to the linear networks (something like 93% vs 89%). The authors should provide comparison of the accuracy levels with other nonlinear DNNs.\n2. There seems to be no mention of the operating frequency of the metasurface DNN. Would it be possible to increase the operating frequency so that the system can be more compact?\n3. The authors should discuss the key factors that set the lower limit of latency, and how it can be further reduced."}, "author_response": "We sincerely appreciate the reviewer for the time and effort invested in reviewing our manuscript and offering a wealth of insightful comments and thoughtful suggestions. These comments and suggestions are extremely valuable to enhance the quality of our manuscript. Following your constructive guidance, we carefully revised the paper to improve its clarity and comprehensibility. Below, we provide our one-by-one responses (in black fonts) to the referees’ comments (in blue fonts).\nRemarks to the Author:\nThe authors have proposed and demonstrated an RF diffractive neural network that features linear and nonlinear layers using RF components. They show how to configure the nonlinearity using electrical bias and gain. Different pattern and posture recognition tasks are demonstrated. It is an interesting implementation, the authors have performed several tests, and the paper is well presented. However, there are several comments that should be addressed before it can be considered for publication:\nResponse:\nThank you very much for your kind support and encouraging comments. We have carefully addressed your questions point by point. Please refer to the details as below.\nComment 1.1:\nHow did the authors choose the frequency of 50 MHz? This seems to make the scaling challenging. The antenna size decreases as frequency goes higher. Also, in the abstract they call the system “microwave” which is not accurate unless it is designed for a much higher frequency.\nResponse:\nWe apologize for the ambiguity in our article. In the original manuscript, the reference to 50 MHz appears in the Methods section under \"Testing the time delay of nonlinear metasurface.\" Here, the 50 MHz frequency specifically denotes the envelope modulation frequency employed for amplitude- shift keying (ASK) modulation of the microwave carrier signal.\nAction Taken:\nTo improve clarity and avoid potential misunderstandings, we have revised both the text in \"Testing the time delay of nonlinear metasurface\" and the accompanying Extended Figure S1 accordingly. Additionally, we have explicitly stated the operational frequency of 5.8 GHz in multiple sections, including the Abstract, Introduction, unit design description, and experimental results, to ensure consistency and precision throughout the manuscript. The specific modifications are as follows:\n[Method]\nTesting the time delay of nonlinear metasurface. The time-delay measurement system employs a 5.8 GHz RF signal generator to produce a continuous radio frequency signal. This signal undergoes amplitude-shift keying (ASK) modulation through a 50 MHz square wave (50% duty cycle) received from a waveform generator via an external trigger interface, thereby generating a modulated RF signal with a square-wave envelope. The modulated signal is then radiated by a transmitting antenna to excite the metasurface. To minimize edge diffraction effects, we implement a coding strategy that activates only the central unit while deactivating the surrounding edge units, which are additionally covered with microwave-absorbing material. For real-time monitoring of the incident signal on the metasurface, a receiving antenna (Antenna 1) is positioned equidistant from the transmitting antenna and the central unit of the metasurface. The RF signal captured by Antenna 1 is transmitted via a coaxial cable to Channel 1 of an oscilloscope. Simultaneously, a second receiving antenna (Antenna 2), placed on the opposite side of the metasurface, detects the transmitted wave and relays the signal through a coaxial cable of identical length to Channel 2 of the oscilloscope. By analyzing the temporal difference between the signal envelopes of the two channels, the transmission delay introduced by the single-layer nonlinear metasurface is precisely quantified.\n[Abstract]\n…In this work, we propose a programmable multilayer full-space nonlinear neural network operating at microwave frequencies (5.8 GHz). …\n[Introduction]\n…Here, we propose a multilayer nonlinear diffractive neural network (MN-DNN) with the programmable and fast rectified linear unit (ReLU) activation function operating at 5.8 GHz. …\n[Architecture of nonlinear metasurface unit]\n…Optimized for 5.8 GHz operation, the patch antenna exhibits a side length dimension of D = 7.57 mm to achieve proper resonance characteristics. …\n[Handwritten digits classification by MN-DNN]\n…The optimization procedure explicitly accounts for the 5.8 GHz horn antenna's field distribution at the input plane under test conditions (Supplementary Note 7), to reduce discrepancies between simulations and measurements.…\n…Both the linear and nonlinear prototypes are excited by a 5.8 GHz horn antenna. …\n[Static posture recognition by MN-DNN]\n…The system employs a standard horn antenna operating at 5.8 GHz to illuminate the test person. Posture information is extracted using an eight-element patch antenna array (5.8 GHz resonant frequency, 3-2-3 arrangement) that samples the scattered field distributions from the human body. …\n[Extended Data]\nTesting of nonlinear metasurface delay\nExtended Figure S1 | Schematic diagram of the system for processing latency measurement. a, The 5.8 GHz RF signal, modulated by a 50 MHz square wave using ASK, is transmitted via the transmitting antenna. Receiving Antenna 1 and Antenna 2 capture the incident and transmitted signals, respectively, routed through matched-length coaxial cables to the oscilloscope. The metasurface transmission delay is derived from the envelope time difference between the two signals. b, ASK-modulated waveform. A 5.8 GHz RF signal amplitude-modulated by a 50 MHz square wave with 50% duty cycle. c, The time delay introduced by the nonlinear layer is defined as the envelope time difference between the transmitted (CH2) and incident (CH1) waveforms.\nComment 1.2:\nIt takes a few seconds for the dynamic posture recognition result to settle. As a fundamental question, if one uses a camera and a digital neural network (on CPU, GPU, …), how long would that take and how does that compare with the proposed architecture? I understand the benefit of not needing RF to digital conversion, but this is important at a system level.\nResponse:\nThank you for your insightful question. As Reviewer 1 rightly point out, system-level speed is indeed a crucial metric. The latency of camera-based posture motion recognition combined with digital neural network (on CPU, GPU, …) is on the order of milliseconds, typically ranging from approximately 43.3 to 1033.3 ms. In contrast, the overall latency of the MN-DNN posture recognition system is 2.48 μs. In the future, by integrating higher-speed ADC modules and FPGAs, the system's overall response latency can be further reduced to approximately 68.5 ns.\nAdditionally, regarding to the output voltage fluctuations observed during dynamic posture recognition in Figure 6, we would like to clarify that these variations are caused by transitional movements during action switching and do not reflect the system's actual processing latency. We sincerely apologize for any potential misunderstanding this may have generated.\nWe would like to address this comment by elaborating on three key aspects that demonstrate the significant latency improvements our posture recognition system achieves compared to traditional systems:\n System latency in camera-based posture recognition (CPU/GPU/Edge computing devices Processing);\n System latency of MN-DNN for posture recognition;\n Strategies for further reduction of delay.\n1) System latency in camera-based posture recognition (CPU/GPU/Edge computing devices Processing)\nThe latency bottlenecks in using cameras combined with digital neural networks primarily stem from two factors: the frame rate of the camera and the processing speed of digital neural network-based post-processing, which can be implemented on GPUs, CPUs, or edge computing devices, as shown in Fig. R1.\nFigure R1. Traditional camera-based motion recognition pipeline: (1) optical image acquisition using RGB/depth cameras, (2) digital neural network processing executed on GPU/CPU/edge computing devices\nCameras Speed\nCommercial cameras used for posture recognition typically operate at 10-60 fps, corresponding to per-frame processing latencies of 16.67-100ms. Table R1 summarizes key specifications (camera models, frame rates, and latencies) from representative studies in this field.\nTable R1. Time delay of cameras\nRepresentative works Camera Frame rate Time delay\nNTURGB+D:A Large Scale Dataset for Microsoft Kinect\n30 fps 33.33 ms\n3D Human Activity Analysis[R1] V2\nReal-Time Human Pose Recognition in Microsoft Kinect\n30 fps 33.33 ms\nParts from Single Depth Images[R2] V1\nMicrosoft Kinect\n30 fps 33.33 ms\nActive Vision for Early Recognition of\nV2\nHuman Actions[R3]\nDUO 3D 60 fps 16.67 ms\nBenchmarking Human Motion Analysis\nMicrosoft Kinect\nUsing\n30 fps 33.33 ms\nV1\nKinect One: an open source dataset[R4]\nComputation Delay in Digital Neural Networks (on CPU/GPU/Edge Devices)\nFor posture recognition using digital neural networks, the inference latency typically ranges from 10 to 1000 ms when running on GPUs, CPUs, or edge devices (e.g., Google Pixel 2’s Snapdragon 835, Nvidia Jetson TX2, etc.). Table R2 summarizes representative studies, listing their respective CPU/GPU/edge device models and the corresponding processing latencies.\nTable R2. Post-processing delay of the digital computing system\nDigital\nRepresentative works Operating platform Net processing\nlatency\nCamera Distance-aware Top-down Approach\nNVIDIA TitanX\nfor 3D Multi-person Pose Estimation from a ResNet-50 147.7 ms\nMaxwell GPU\nSingle RGB Image (ICCV 2022) [R5]\nSMAP: Single-Shot Multi-person Absolute 3D NVIDIA GTX1080Ti\nHourglass 62.3 ms\nPose Estimation (ECCV 2020) [R6] GPU\nSingle-Stage is Enough: Multi-Person Absolute\nNVIDIA V100 GPU HRNet 55.6 ms\n3D Pose Estimation (CVPR 2022) [R7]\nDistribution-Aware Single-Stage Models for\nResNet-50 +\nMulti-Person 3D Pose Estimation (CVPR NVIDIA V100 GPU 75 ms\nFPN\n2022) [R8]\nNVIDIA V100 GPU 10 ms\nTiny Video Networks (Applied AI Letters 2022)\nTVN-1\nIntel Xeon 2.9GHz\n[R9]\n37 ms\nCPU\nReal-time 2D Multi-Person Pose Estimation on\nIntel Core i7-6850K Dilated\nCPU: Lightweight OpenPose (ICPRAM 2019) 38.5 ms\nCPU MobileNet v1\n[R10]\nMoViNets: Mobile Video Networks for\nEfficient Video Recognition (CVPR 2021) Intel x86 Xeon CPU 3D CNN 10 ms\n[R11]\nMobile CPU core Heatmap-\nBlazePose: On-device Real-time Body Pose\n(Google Pixel 2’s Regression 32.3 ms\ntracking[R12]\nSnapdragon 835) Hybrid\nMulti-Person 3D Pose Estimation in Mobile\nLightweight\nEdge Computing Devices for Real-Time Raspberry Pi 4 1000 ms\nResidual CNN\nApplications (ICOIN 2023) [R13]\nA Distributed Real-time 3D Pose Estimation\nFramework based on Asynchronous Edge Proposals\nNvidia Jetson TX2\n50 ms\nMultiviews (KSII Trans Internet Inf Syst 2023) Triangulation\n[R14]\n2) System latency of MN-DNN for posture recognition\nThe latency in the MN-DNN recognition system consists of two parts: the transmission delay in the MN-DNN and the output detection delay for identifying the location of peak energy, as shown in Fig. R2.\nFigure R2. Latency composition analysis for MN-DNN-based posture recognition. a, Propagation delay through MN-DNN architecture. b, Detection delay at output plane. c, Detailed MN-DNN propagation components: air transmission and layer-wise delays through 3 linear and 3 nonlinear layers. d, Output detection components: RF detector response time and MCU processing delays (ADC conversion + voltage comparison).\nMN-DNN propagation delay\nThe propagation delay in MN-DNN primarily consists of three components: the delay introduced by the nonlinear layers, the delay in the linear layers, and the propagation delay through air, as illustrated in Fig. R2c.\na. Nonlinear layers delay. The system comprises three nonlinear layers. The complete activation of each nonlinear layer introduces a latency of 17.7 ns (refer to response to Comment 1.3). Consequently, the total nonlinear delay generated by the cascaded nonlinear layers amounts to 53.1 ns (3 layers × 17.7 ns/layer).\nb. Linear layers delay. The MN-DNN also consists of three linear layers, with the per-layer delay calculated by t = h/v, where h = 4.8 mm represents the thickness of the dielectric medium in the linear layer, and v = 1.43 × 10⁸ m/s denotes the propagation speed of EM waves in the FR4 substrate (with a relative permittivity of 4.4). This yields a single-layer delay of 33.6 ps, leading to a cumulative linear delay of 100.8 ps across three linear layers.\nc. Propagation delay through air. In the air transmission segment, the EM wave travels a distance of approximately 0.7 meters between the human body and the output plane at the speed of light, introducing an additional delay of 2.33 ns.\nIn summary, MN-DNN propagation delay amounts to 55.53 ns (53.1 ns + 100.8 ps + 2.33 ns).\nOutput plane detection delay\nThe output plane's detection latency comprises three components: RF detector delay, MCU ADC sampling time, and multichannel comparison latency.\na. RF detector delay. The LTC5564 RF detectors are employed to measure the received energy from each antenna in the output plane, introducing a delay of 7ns.\nb. ADC sampling time. ADC conversion is implemented through the analog input ports of the Arduino Due R3 MCU. The ADC operates at 500 KSPS (2.0 μs/sample) to maintain stable performance.\nc. Multichannel comparison latency. The system conducts eight-channel voltage comparisons to identify the maximum value through seven sequential operations, obtaining the classification result. Operating at 84 MHz (11.9 ns clock cycle), each comparison requires five clock cycles (59.5 ns), yielding a total comparison latency of 416 ns (35 clock cycles).\nIn summary, the output plane detection delay amounts to 2.423μs (7ns + 2.0 μs + 416 ns).\nThe total system delay for posture recognition utilizing the MN-DNN framework is presented in the Table R3, amounting to 2.48 μs.\nTable R3. System latency of MN-DNN-based posture recognition\nEvents MN-DNN propagation delay Output Plane detection delay Overall\nDelay 55.53 ns 2.407 μs\n2.48μs\n3) Strategies for further reduction of delay\nThe analysis reveals that ADC sampling and multichannel comparison constitute the primary latency sources in limb detection systems. Substantial improvements can be achieved by implementing GHz- speed ADC modules (e.g., ADI ADC12DJ3200/ADC12J4000, ~1 ns latency) and replacing MCUs with high-performance FPGAs (Intel Stratix 10/Xilinx Kintex Ultrascale) for rapid 8-channel signal processing (3-5 ns latency). Fig.R3 presents the optimized detection workflow.\nFigure R3. Optimized output-plane detection system with latency reduction. Utilizing high-speed ADC and FPGA components for minimized delay.\nIn summary, the optimized MN-DNN system achieves a total latency of 66.5 to 68.8 ns, as shown in Table R4.\nTable R4. Optimized system latency of MN-DNN\nOptimized output Plane\nEvents MN-DNN propagation delay Overall\ndetection delay\nDelay 55.53 ns 11~13 ns 66.5~68.5 ns\n4) Comparative latency analysis of multiple systems\nFor comparison, we evaluate the latency performance of traditional (30fps camera-based) and MN- DNN implementation. System latency comparison is shown in Table R5.\nTable R5. System latency comparison\nRepresentative works Operating platform Total system latency\nCamera-based posture recognition\n(CPU/GPU/Edge computing devices CPU/ GPU/edge devices 43.3 ~1033.3 ms\nProcessing) [R5]-[R14]\nMN-DNN (this paper) MN-DNN 2.48 μs\nIn summary, we employ MN-DNN to directly process EM waves for posture classification with a total system latency of 2.48 μs. This represents a three-order-of-magnitude reduction in latency compared to conventional camera-based approaches reliant on GPU/CPU processing (from milliseconds to microseconds). This advancement eliminates the frame rate and processing delay constraints inherent in existing methods. By integrating higher-speed ADC modules and FPGAs, the MN-DNN’s system latency can be further reduced to the nanosecond (ns) range.\nReferences:\n[R1] Shahroudy A, Liu J, Ng T-T, Wang G. NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)) (2016).\n[R2] Cheng X, Zhang L, Tang Y, Liu Y, Wu H, He J. Real-Time Human Activity Recognition Using Conditionally Parametrized Convolutions on Mobile and Wearable Devices. IEEE Sens J 22, 5889-5901 (2022).\n[R3] Wang B, Huang L, Hoai M. Active Vision for Early Recognition of Human Actions. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)) (2020).\n[R4] Leightley D, Yap MH, Coulson J, Barnouin Y, McPhee JS. Benchmarking human motion analysis using kinect one: An open source dataset. In: 2015 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA)) (2015).\n[R5] Moon G, Chang JY, Lee KM. Camera Distance-Aware Top-Down Approach for 3D Multi- Person Pose Estimation From a Single RGB Image. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV)) (2019).\n[R6] Zhen J, et al. SMAP: Single-Shot Multi-person Absolute 3D Pose Estimation. In: Computer Vision – ECCV 2020 (eds Vedaldi A, Bischof H, Brox T, Frahm J-M). Springer International Publishing (2020).\n[R7] Jin L, et al. Single-Stage is Enough: Multi-Person Absolute 3D Pose Estimation. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)) (2022).\n[R8] Wang Z, Nie X, Qu X, Chen Y, Liu S. Distribution-Aware Single-Stage Models for Multi- Person 3D Pose Estimation. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)) (2022).\n[R9] Piergiovanni AJ, Angelova A, Ryoo MS. Tiny Video Networks. Applied AI Letters 3, (2022).\n[R10] Osokin D. Real-time 2D Multi-Person Pose Estimation on CPU: Lightweight OpenPose. In: Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods) (2019).\n[R11] Bazarevsky V, Grishchenko I, Raveendran K, Zhu TL, Zhang F, Grundmann MJA. BlazePose: On-device Real-time Body Pose tracking. abs/2006.10204, (2020).\n[R12] Imtiaz Hossain M, Akhter S, Delowar Hossain M, Seon Hong C, Huh E-N. Multi-Person 3D Pose Estimation in Mobile Edge Computing Devices for Real-Time Applications. In: 2023 International Conference on Information Networking (ICOIN)) (2023).\n[R13] A Distributed Real-time 3D Pose Estimation Framework based on Asynchronous Multiviews. KSII Trans Internet Inf Syst 17, (2023).\nAction Taken:\n[Dynamic posture recognition by MN-DNN]\nThe total system latency is 2.48 μs, comprising both the propagation delay through the MN-DNN and the processing delay incurred by the output detection. This represents an at least three-order-of-magnitude reduction in latency compared to conventional camera systems coupled with digital neural network post-processing, which typically operate at millisecond-level latencies. By implementing a higher-speed output detection scheme (e.g., utilizing high-speed ADC modules and high-performance FPGAs), the total system latency could be further reduced to 66.5–68.5 ns (see Supplementary Note 27 for details).\n[Supplementary Note 27: Latency analysis and optimization strategies for posture recognition systems]\nThe aforementioned results and analyses have been included in Supplementary Note 27: Latency analysis and optimization strategies for posture recognition systems.\nComment 1.3:\nThe adder that is used in the experimental setup uses TLV9001 opamp with a gain bandwidth product of 1 MHz (1us time constant). How can the system achieve a latency of 4.5ns as the adder seems to be involved in the forward path?\nResponse:\nWe appreciate the reviewer's insightful comments regarding the system latency issue. Under initial experimental conditions (with a bias voltage V of 1 V), the nonlinear metasurface operates in BIAS linear transmission mode. The measured delay of 4.5 ns in this case reflects the EM wave propagation delay through the metasurface in its fully conductive state, not the nonlinear activation delay. We acknowledge that this measurement approach may inadvertently lead to a misinterpretation of the nonlinear layer's response speed, and we apologize for any confusion this may cause. To resolve this issue, we implement design optimizations using advanced high-speed components. Specifically, we select the THS3201 operational amplifier (measured delay: 0.7 ns), the HMC326S8G RF amplifier (measured delay: 10 ns), and the LTC5564 RF detector (measured delay: 7 ns). Experimental results confirm that the total delay of the optimized single-layer nonlinear metasurface is 17.7 ns, clearly validating the effectiveness of our approach. This optimization significantly reduces the delay while maintaining the original nonlinear working principle and performance characteristics.\nThe latency issue will be examined from four perspectives:\n(1) Original activation latency in the nonlinear layer\n(2) Optimized design of the faster nonlinear layer\n(3) Experimental validation of active devices and nonlinear circuit latency\n(4) Experimental study of faster nonlinear metasurface latency performance\n1) Current activation latency in the nonlinear layer\nThrough further investigation and testing (as shown in Fig. R4), we observe that the original design exhibits a nonlinear activation delay of 2.3 μs under nonlinear conditions (e.g., at 0 V bias).\nFigure R4. Original nonlinear metasurface activation delay measurement system and results. a, Schematic of the experimental setup. An ASK-modulated RF signal is generated and irradiated onto the metasurface sample via a transmitting horn antenna. Only the central unit cells of the metasurface are activated (the remaining cells are covered with microwave absorbers to suppress spurious signals). The receiving system consists of: Rx Antenna 1 (front side) detecting the incident signal (Oscilloscope Ch. 1), and Rx Antenna 2 (back side) capturing the transmitted signal (Oscilloscope Ch. 2). The time difference between the two channels corresponds to the nonlinear activation delay (The coaxial lines delay is pre-calibrated). b, Photograph of the receiving antenna on the metasurface backside. c, Measured time delay. The nonlinear activation delay is experimentally determined to be 2.3 μs.\n2) Optimized design of the faster nonlinear layer\nTo further enhance the activation speed of the nonlinear metasurface, we optimize its design by replacing key active components while fully preserving the original nonlinear characteristics and functional architecture (detector-voltage adder-RF amplifier). Specifically, we employ the THS3201 operational amplifier[R14] (with a gain bandwidth product of 1.8 GHz and a slew rate of 6700 V/µs), the LTC5564 RF detector[R15], and the HMC326MS8G RF amplifier[R16]. Fig. R5 presents the detailed schematic diagrams of these components.\nNotably, the optimized nonlinear metasurface unit retains the same architecture and ReLU nonlinearity mechanism as the original design. The enhancement only replaces the original devices (operational amplifier, RF detector and RF detector) as faster active devices (see Figure R6 for the revised structure), while all functional characteristics remain unchanged from the original implementation. The optimized nonlinear metasurface retains all key features of the original design: the RF-based ReLU-like activation mechanism, programmable threshold/slope tuning, and characteristic nonlinear response. This functional equivalence ensures our previously demonstrated performance enhancements (in image classification and in action recognition) remain valid, while achieving a faster response speed.\nFigure R5. Schematic diagrams of the high-speed components. a, Schematic of the RF amplifier (HMC326MS8G). b, Schematic of the RF detector (LTC5564). c, Schematic of the operational amplifier (THS3201).\nFigure R6. Operation principle of the optimized nonlinear metasurface unit. The front patch antenna receives incident waves, which are split by a power divider. One branch is converted to DC by an RF detector, then combined with a preset bias voltage to dynamically control the gain of the second branch's RF amplifier. The processed signal is finally radiated via the rear transmitting antenna. The operational flow remains unchanged from the original design, with only faster active components being implemented.\n3) Experimental validation of active devices and nonlinear circuit latency\nTo validate the improvements, we design and fabricate test boards for each device and conduct detailed delay measurements. The testing procedures and results for individual devices appear in Fig.R7 to R9. The results show that the operational amplifier exhibits a delay of approximately 0.7 ns (Fig.R7), the RF amplifier demonstrates a delay of about 10 ns (Fig.R9), and the RF detector shows a delay of around 7 ns (Fig. R8).\nAdditionally, the joint delay measurement results for the complete active nonlinear circuit appear in Fig. R10. The test results indicate that the overall delay of the nonlinear circuit is 17.7 ns. A summary of the test data appears in Table R6. These enhancements ensure that the system's response speed significantly improves while maintaining its original nonlinear characteristics, reducing the nonlinear response time to the nanosecond level.\nFigure R7. THS3201 operational amplifier test system and experimental results. a. Schematic diagram of the THS3201 operational amplifier; b. Photograph of the THS3201 evaluation board; c. Schematic diagram of the time delay test setup (Test method: The signal generator output is split into two paths, one is directly connected to oscilloscope Channel 1 as the reference signal, while the other is fed through the operational amplifier and superimposed with a DC bias voltage before being connected to oscilloscope Channel 2. The propagation delay of the operational amplifier is determined by comparing the time delay difference between the two signals. Note: All cable transmission delays and PCB trace delays have been calibrated); d. Photograph of the experimental setup for the time delay test; e. Test results of the input and output voltage waveforms (measured propagation delay: 0.7 ns).\nFigure R8. LTC5564 RF detector test system and experimental results. a. Schematic diagram of the LTC5564 RF detector; b. Photograph of the LTC5564 evaluation board; c. Schematic diagram of the time delay test setup (Test method: The signal generator produces an ASK-modulated RF signal, which is split into two paths via a power divider, one is directly connected to the oscilloscope as the reference signal, while the other is demodulated by the RF detector before being fed into the oscilloscope. The detector delay is obtained by comparing the time delay difference between the two signals. Note: All cable transmission delays have been calibrated); d. Photograph of the experimental setup for the time delay test; e. Test results of the input and output signal waveforms (measured detection delay: 7 ns).\nFigure R9. HMC326MS8G RF amplifier test system and experimental results. a. Schematic diagram of the HMC326MS8G RF amplifier; b. Photograph of the HMC326MS8G evaluation board; c. Schematic diagram of the switching delay test setup (Test method: The signal generator output is split into two paths, one is directly connected to the oscilloscope as the reference signal, while the other is routed through the RF amplifier whose ON/OFF state is controlled by modulating its control voltage. The amplifier output is then fed into a second oscilloscope channel. The switching delay of the RF amplifier is determined by comparing the time difference between the reference signal and the amplifier output. Note: All cable transmission delays have been calibrated); d. Experimental setup for the switching delay test; e. Test results of the input and output signal waveforms (measured switching delay: 10 ns).\nFigure R10. Joint test system and results for nonlinear active circuits. a. Schematic diagram of the test system (Signal flow: The ASK-modulated RF signal is divided into three paths via a power divider: (1) The reference signal is directly connected to the oscilloscope; (2) The detected signal is demodulated by the LTC5564 detector, then superimposed with a bias voltage by the THS3201 operational amplifier; (3) The synthesized voltage controls the ON/OFF state of the HMC326MS8G RF amplifier, whose output is connected to another oscilloscope channel); b. Photograph of the integrated active circuit evaluation board (The board integrates three key components: RF detector, RF amplifier, and operational amplifier, forming the active circuit module for faster nonlinear metasurface unit); c. Photograph of the Experimental test setup; d. Comparison of input and output RF energy waveforms (measured total system delay: 17.7 ns).\nTable R6. Time delay in faster nonlinear devices response\nElements RF detector Voltage adder RF amplifier Overall\nTime delay 7 ns 0.7 ns 10 ns 17.7 ns\n4) Experimental study of faster nonlinear metasurface latency performance\nAfter completing independent time-delay verification of active devices and nonlinear activation circuits, we fabricate an optimized nonlinear metasurface prototype based on the improved design. The prototype maintains the same architecture as the previous version but integrates higher-speed active components (LTC5564 RF detector, THS3201 operational amplifier, and HMC326MS8G RF amplifier). As Fig.R11 shows, the metasurface 's front side integrates the LTC5564 detector, while the back side incorporates the THS3201 operational amplifier and HMC326MS8G RF amplifier, preserving the original layout while replacing components with faster alternatives.\nFigure R11. Physical images and measured performance of the new fast nonlinear metasurface. a, Front- view photograph of the new fast nonlinear metasurface. b, Rear-view photograph of the new nonlinear metasurface. c, Measured time-delay characteristic curve. d, Zoomed-in view of the time-delay characteristic (measured time delay: 17.7 ns).\nWe evaluate the new metasurface 's time-delay characteristics using the same test method described previously (Fig.R4). The results in Fig.R11c and the more detailed Fig.R11d reveal an overall input-to-output delay of approximately 17.7 ns, matching the circuit-level test results. These results demonstrate successful realization of a metasurface with nanosecond-scale nonlinear activation response, which maintains the original nonlinear characteristics while significantly improving response speed.\nReferences:\n[R14] THS3201: 1.8-GHz, LOW DISTORTION, CURRENT-FEEDBACK AMPLIFIER. Datasheet, Texas Instruments, Rev. C, 2009. [Online]. Available: https://www.ti.com.cn/cn/lit/ds/symlink/ ths3201.pdf?ts=1753861828591&ref_url=https%253A%252F%252Fwww.ti.com.cn%252Fproduct %252Fcn%252FTHS3201.\n[R15] LTC5564: UltraFast™ 7ns Response Time 15GHz RF Power Detector with Comparator. Datasheet, Analog Devices, Inc., Rev. B, 2018. [Online]. Available: https://www.analog.com/ media/en/technical-documentation/data-sheets/5564fc.pdf\n[R16] HMC326MS8G: GaAs InGaP HBT MMIC DRIVER AMPLIFIER. Datasheet, Analog Devices, Inc., Rev. 0, 2013. [Online]. Available: https://www.analog.com/media/en/technical-documentation/ data-sheets/hmc326.pdf\nAction Taken:\n[Abstract]\nExperimental results indicate that the proposed nonlinear architecture, with optimized device selection, achieves a nanosecond-scale delay (17.7ns), outperforming optoelectronic conversion- based nonlinear approaches by several orders of magnitude in speed.\n[Introduction]\nWith optimized high-speed device selection, the engineered single-layer nonlinear metasurface achieves nanosecond-scale delays (17.7 ns), representing a substantial reduction compared to the millisecond (ms) and microsecond (µs) latency typically associated with conventional nonlinear optoelectronic conversion processes.\n[Principle of the MN-DNN]\nFig. 1 | Nonlinear diffractive neural networks (DNNs) for multi-target recognition and processing. a Operations of the nonlinear DNNs, including dataset-based image recognition and real-time direct human posture classification. The information-carrying electromagnetic waves are processed through MN-DNN with three linear layers and three nonlinear layers. The classification outcome is indicated by the maximum energy distribution in the designated regions on the output plane. Each nonlinear unit is integrated with an RF detector, amplifier, and voltage adder. b Transmittance characterization of the nonlinear layer, showing no output for weak inputs and a linear increase in output energy for strong inputs, mimicking the ReLU activation function. The nonlinear activation’s threshold and slope are dynamically adjustable. c Time delay characteristics of the nonlinear metasurface. The time delay is 17.7 ns for the single-layer nonlinear metasurface. d, e, f The network capabilities for processing images from MNIST and Fashion-MNIST, and real-time classification of eight posture categories.\nHowever, our architecture achieves a reduced latency of 17.7 ns (Fig. 1c), marking a substantial reduction in the delay and underscoring its importance for real-time processing.\n[Architecture of nonlinear metasurface unit]\nFig. 2 | Nonlinear metasurface unit configuration. a The nonlinear metasurface unit, featuring four metal and three dielectric layers, with F4B as the top and bottom dielectrics and FR4 as the middle layer. The unit functions as a summation, nonlinear activation, and amplitude modulation device. b Front and back views of the nonlinear metasurface unit, featuring an octagonal receive antenna on the front with an integrated RF detector, and on the back, a corresponding octagonal transmit antenna accompanied by an RF amplifier and voltage adder. c Operational diagram of the nonlinear metasurface unit, showing the receive antenna directing EM energy to the back for amplification and a portion to the RF detector for rectification. The rectified voltage and an external bias voltage can modulate the RF amplifier’s DC control voltage V , with V controlling the gain. The amplified RF energy CTRL CC is then radiated by the transmit antenna. d The output DC voltage of the RF detector is positively correlated with the incident power density on the nonlinear metasurface. e The gain of the RF amplifier is jointly determined by DC voltages V and V . f The RF amplifier’s gain exhibits a minimal value at low V , effectively cuts off, and subsequently increases to a saturation point with further V increments. g The RF amplifier’s gain is minimal at low V , effectively cuts off, and then rises to a plateau as the additional V increases. h For V > 1.2V, the nonlinear metasurface unit is operated linearly, with the output energy directly proportional to the input energy, and gain escalating with V . i The transmission coefficient of the nonlinear metasurface unit rises from a minimal value at low incident intensities to a plateau at higher levels, with V reducing the energy threshold for transition from the cutoff to transmission. j The transmission coefficient of the nonlinear metasurface unit escalates with the incident energy, reaching a plateau, and is enhanced by increased V at constant input energy levels. k With V = 1V and V = 5V, the nonlinear metasurface unit exhibits zero transmission for the incident energies below 1 mW and a linear increase above this threshold, akin to a ReLU function. l The unit’s nonlinear activation threshold is inversely related to V . m The unit’s gain is directly proportional to V .\nBIAS CC\nThe RF detector’s output voltage V is positively correlated with the input RF energy, as OUT shown in Fig. 2d…. Fig. 2g indicates that low V values result in a transmission coefficient below CTRL -10dB, halting the transmission. Above 1.2V, V initiates the signal amplification, maintaining a CTRL stable gain despite further increases. Fig. 2h shows that with a high external bias voltage V (e. g., BIAS V = 1.5V), the amplifier’s V surpasses the amplification threshold, independent of the BIAS CTRL detector’s output voltage V . …We note that the detector’s V and the amplifier’s V are low OUT OUT CTRL when the incident power on the metasurface is below 1 mW, as demonstrated in Fig. 2k, leading to amplifier cutoff and zero output from the metasurface unit. Beyond 1 mW, the voltages will be increased, activating the amplifier’s linear amplification and causing the metasurface unit’s output power to rise linearly with the input power, which emulates the ReLU function in neural networks, in this scenario, V is 0.5 V and V is 5 V. …This nonlinear activation function offers 17.7 ns latency BIAS CC in the measurements (details see Supplementary Note 3 and Extended Data 1).\n[Supplementary Note 1: Schematic details of nonlinear metasurface units]\nFigure S1a illustrates the schematic of the RF amplifier HMC326SM8G. This eight-port integrated circuit receives RF signals at the RFIN port, amplifies them, and routes the output to the RF port. OUT The gain is regulated by the combined application of external DC voltages V and V . Figure S1b CC CTRL presents the circuit diagram of the RF detector LTC5564. The LTC5564 is a 16-port integrated circuit that receives RF energy at the RF port for power detection, with the output DC voltage V IN OUT showing a positive correlation with the detected energy level. The chip is powered by the V port. DD DC ports are equipped with paralleled filtering capacitors to ensure stable voltage regulation. Figure S1c presents the circuit diagram of the voltage adder THS3201. The eight-terminal THS3201 operational amplifier chip is powered by a positive DC supply voltage VS+ (+7.5 V) connected to V+ and a negative DC supply voltage VS- (-7.5 V) connected to V-. V and V are each connected in bias to the IN+ terminal through 50-ohm resistors, while IN- is connected to the V terminal via a 768- out ohm resistor, which is also shunted to ground with an additional 768-ohm resistor.\nBased on the characteristics of operational amplifiers, it is known that:\n𝑉 = 𝑉 ∙\n𝑅5\n+𝑉 ∙\n𝑅6\n(S1)\n𝐼𝑁+ 𝑖𝑛 𝑏𝑖𝑎𝑠\n𝑅5+𝑅6 𝑅5+𝑅6\n𝑉 = (1+\n𝑅𝑓)∙V\n(S2)\n𝑜𝑢𝑡 𝑖𝑛\n𝑅𝑔\nGiven that we have set R = R = 50 Ω, R = R =768 Ω, it follows that the output voltage V is equal 5 6 f g out to the sum of the input voltage V and the bias voltage V ,\nin bias\n𝑉 = 𝑉 +𝑉 (S3)\n𝑜𝑢𝑡 𝑖𝑛 𝑏𝑖𝑎𝑠\nFigure S1d depicts the equivalent circuit of a nonlinear metasurface element. The RF energy, post-forward coupling, bifurcates: the primary portion is amplified by the RF amplifier and then radiated by the backward coupling structure, while a minor fraction is diverted to the RF detector for power detection. The detector's output, a DC voltage V is fed into the IN+ terminal of the voltage adder, OUT where it combines with the DC bias V . This sum is applied to the V of the amplifier, BIAS CTRL modulating its gain in response to the incident energy, thus dynamically adjusting the element's transmission coefficient.\nSupplementary Figure S1 | Schematic diagrams of the high-speed components. a, Schematic of the RF amplifier (HMC326MS8G). b, Schematic of the RF detector (LTC5564). c, Schematic of the operational amplifier (THS3201).\nSupplementary Figure S2 | Operation principle of the optimized nonlinear metasurface unit. The front patch antenna receives incident waves, which are split by a power divider. One branch is converted to DC by an RF detector, then combined with a preset bias voltage to dynamically control the gain of the second branch's RF amplifier. The processed signal is finally radiated via the rear transmitting antenna. The operational flow remains unchanged from the original design, with only faster active components being implemented.\n[Supplementary Table S2 | Performance comparison of different nonlinearity mechanisms in diffractive neural networks.]\n[Supplementary Note 3: Delay time test]\nThe aforementioned results and analyses have been included in [Supplementary Note 3: Delay time test].\nComment 1.4:\nThe images are binarized. What would happen if the image resolution were higher than just one bit? How would the systems operate? This is an important point as a more realistic application requires higher resolution.\nResponse:\nWe thank the reviewer for this insightful and forward-looking question. To achieve multi-bit amplitude input functionality, our system architecture can be enhanced by integrating an digital-to-analog converter (DAC) module into the control circuitry of the input-layer metasurface. This implementation allows for precise and continuous gain control of individual units in the input-layer metasurface, supporting high-resolution amplitude modulation essential for more realistic applications. We address the issue from two complementary perspectives:\n Input-layer metasurface hardware is compatible with multi-bit control schemes.\n Analysis of prospective methodologies for achieving such multi-bit control in future systems.\n1) Input-layer metasurface hardware is compatible with multi-bit control schemes\nWe employ a gain-programmable metasurface as the input layer, whose structure is illustrated in Fig. R12a. Each unit consists of a front metallic patch, a back metallic patch, and an RF amplifier connecting them. By adjusting the DC bias voltage of the amplifier (0.1V-3V), we achieve continuous gain tuning from -16 dB to 13 dB at the operating frequency, corresponding to a total transmission coefficient tuning range of 29 dB, as shown in Fig.R12b. This multi-bit amplitude modulation characteristic endows the metasurface with continuously tunable transmission coefficients, thereby providing the system with continuous grayscale input encoding capability. Each unit is equipped with an external control interface, enabling continuous and independent gain tuning of individual RF amplifiers, as shown in Fig.R12c.\nFigure R12 | Schematic diagram of the input layer with its parametric characteristics. a, The input layer is constructed using an amplitude-programmable metasurface. Each unit integrates a receiving antenna on the front side, a transmitting antenna on the back side, and an RF amplifier linking the two. The amplifier gain is adjustable via the supply voltage VCC. b, Variation of the transmission coefficient at the resonant frequency with respect to the supply voltage. As the voltage increases from 0.1 V to 3 V, the transmission coefficient improves from −16 dB to 13 dB, demonstrating a dynamic tuning range of 29 dB. c, Circuit wiring design of the input-layer metasurface control system, enabling independent voltage regulation for each unit.\n2) Analysis of prospective methodologies for achieving such multi-bit control in future systems\nTo enable continuous and precise amplitude modulation of the input plane, we propose a high- precision control scheme utilizing a DAC module. The system employs an FPGA or MCU as the main controller, coupled with a high-resolution DAC module (such as AD5328) to form a complete voltage regulation system. As depicted in Fig. R13, the implementation involves three key steps: (1) The FPGA or MCU generates digital control signals, transmitted via standard interfaces (e.g., SPI/I2C) to a 16-bit DAC module; (2) The DAC module converts the digital signals into analog voltages; (3) This precisely controlled voltages are applied to the metasurface unit’s control ports. This precision voltage control approach enables high-resolution multi-bit modulation of the transmission coefficient for each unit in input layer.\nFigure R13 | Schematic of the high-precision input control system. The system generates digital control signals via an FPGA or MCU, which are then converted into analog voltage signals through a high-precision DAC module. These signals are applied to the metasurface input layer units, enabling multi-bit precise modulation of the transmission coefficient.\nFinally, it should be noted that our system supports two distinct input modalities. The first is the encoded input mode, which is the primary focus of this question. In the future, this mode can be enhanced by integrating a DAC module to achieve high-precision multi-bit input. The second is the direct electromagnetic wave processing mode, where the MN-DNN can directly interpret electromagnetic wave signals in space, eliminating the need for complex encoding conversion processes. This mode enables rapid classification and recognition in applications such as posture identification. These two modalities are tailored for different application scenarios and together form a comprehensive input solution for the system.\nAction Taken:\n[Discussion]\nFor image classification tasks, future architectural improvements could target two key aspects to enhance recognition accuracy and adapt to a broader range of tasks: Firstly, integration of a digital-to-analog converter (DAC) with the input-layer metasurface to enable continuous grayscale inputs, thereby improving information fidelity and subsequent feature extraction accuracy (Supplementary Note 30) …\n[Supplementary Note 30: Input layer metasurface and method for continuous grayscale input]\nThe aforementioned analyses have been included [Supplementary Note 30: Input layer metasurface and method for continuous grayscale input].\nComment 1.5:\nThe linear and nonlinear networks are trained separately. Is this optimal? Could the authors train the whole network together?\nResponse:\nWe sincerely apologize for any potential confusion caused by our previous description and would like to provide a more precise clarification regarding our network architecture and training methodology. In our training protocol, all parameters of the nonlinear network (including those from three linear layers and three nonlinear layers) are jointly optimized in an end-to-end manner. We deliberately design and independently train two structurally distinct networks: a nonlinear network and a linear network. The linear network serves as a control benchmark for performance comparison. Crucially, these two systems remain entirely independent throughout the training process, with no parameter sharing or mutual interference between them.\nFor the nonlinear network, all parameters are trained together, totaling 1458: 484 for each linear layer and two for each nonlinear layer (threshold and slope), as shown in Fig.R14. Specifically, the nonlinear network employs a composite architecture consisting of three linear layers interleaved with three nonlinear layers. All trainable parameters in this network, including the 14×14 unit phase shift parameters in each linear layer and the activation function thresholds and slope parameters in the nonlinear layers, undergo end-to- end joint optimization through a unified error backpropagation algorithm. This ensures that the parameters of both the linear and nonlinear layers update synchronously and co-optimize as an integrated system. In contrast, the linear network contains only three linear layers, with its trainable parameters limited to the 14×14 unit phase shifts in each layer. These parameters are also trained independently using backpropagation.\nFor example, in the MNIST classification task, we strictly maintain the training independence of both networks, subjecting each to a complete training process to obtain their respective optimal parameter configurations (as shown in Fig.R14a). This fully independent training approach enables an objective comparison of the two architectures’ performance in image recognition tasks.\nFigure R14 | Training mechanism of linear and nonlinear diffractive neural networks, with each network trained independently. A, Training process of a nonlinear diffractive neural network on the MNIST dataset, where error backpropagation jointly optimizes three linear layers and three nonlinear layers. b, Training process of a linear neural network on the MNIST dataset, using error backpropagation to train three linear layers.\nAction Taken:\n[Handwritten digits classification by MN-DNN]\n…The nonlinear network consists of 1,458 trainable parameters, with 484 parameters per linear layer and 2 parameters (threshold and slope) per nonlinear layer, yielding a total of 484 × 3 + 2 × 3 = 1,458, and all parameters are trained together …The linear and nonlinear networks remain entirely independent throughout their design and training phases, with no shared parameters or cross-influence.\nComment 1.6:\nHow much is the total power consumption of the system? How does it compare with other approaches?\nResponse:\nThank you for your question. We have supplemented the calculation of the system’s overall power consumption and conducted a comparative analysis with other DNN architectures. The power consumption of the nonlinear meta-unit is 0.429W, with the total power consumption of the system is 252.352 W. The proposed system achieves an energy efficiency of 0.25 TOPS/W, which is competitive with existing approaches. Further power reduction can be attained by integrating ultra-low-power components such as the MAX2659 RF amplifier (power consumption: 6.56 mW), LT5537 RF detector (40.5 mW), and LPV821 operate amplifier (2.844 μW), potentially reducing total power consumption by~90%. Notably, our architecture exhibits strong scaling advantages. When scaled to 56×56 nonlinear units per layer (with linear layers expanding to 88×88), energy efficiency improves to 4 TOPS/W. This scalability makes our approach particularly compelling for large-scale neural network deployment where power efficiency is critical.\nThe discussion will be focused on two aspects:\n The overall power consumption and energy efficiency of the system,\n Comparisons with other DNN networks.\n1) The overall power consumption and energy efficiency of MN-DNN\nPower Consumption of Nonlinear Metasurfaces\nEach nonlinear unit exhibits a total power dissipation of approximately 0.429W, which consists of 150mW for the RF amplifier, 53mW for the RF detector, and 226 mW for the voltage adder, as shown in Table R7.\nTable R7. Power consumption of the nonlinear unit\nTotal\nDevices RF amplifier RF detector Voltage Adder\n（Per Nonlinear Unit）\nPower Consumption 150mW 53mW 226mW 0.429W\nFigure R15. Power consumption measurement of the nonlinear metasurface unit in MN-DNN.\nThe measured power consumption of the nonlinear unit is 0.429 W, as shown in Fig. R15, representing the cumulative power consumption across three voltage channels. Based on this measurement, we calculate that a single-layer nonlinear metasurface (comprising 14×14 units) consumes 84.084 W, while the three-layer nonlinear metasurface configuration exhibits a total power consumption of 252.25 W.\nTotal System Power Consumption\nThe total system power consumption comprises not only the nonlinear metasurfaces' power dissipation but also includes contributions from the RF signal source (0.1 W) and the MCU/control circuitry (approximately 0.002 W). The aggregate power consumption of the complete system is therefore calculated to be approximately 252.352W, as shown in Table R8.\nTable R8. Power consumption of the MN-DNN system\nNonlinear Total Power\nDevices RF Source MCU/Control Borad\nmetasurfaces Consumption\nPower Consumption 252.25 W 0.1W 0.002W 252.352W\nSystem Energy Efficiency\nThe computation of total network operations necessitates a comprehensive analysis encompassing both propagation operations and phase/amplitude modulation computations, involving phase modulation in linear layers and amplitude modulation in nonlinear layers. The real-valued operation count for propagation operations in each layer is determined by its input (N) and output (M) dimensions, requiring N×M complex multiplications and (N-1)×M complex additions, which translate to 4×N×M and 2×(N-1)×M real operations respectively.\nThe proposed network architecture features a 14×14 input layer, three alternating 22×22 linear phase-modulation layers and 14×14 nonlinear layers, followed by a 10-node output layer. Through systematic calculation, the three linear layers collectively account for 1,704,648 real operations (568,216 per layer), while the three nonlinear layers contribute 1,706,376 real operations (568,792 per layer), with the output layer adding 11,740 operations.\nFor phase/amplitude modulation operations, each layer comprising H units requires 6H real operations. Specifically, the linear layers (484 units per layer) account for 8,712 total operations (2,904 per layer), and the nonlinear layers (196 units per layer) contribute 3,528 operations (1,176 per layer). The aggregate computation yields a total operational count of 3,435,004 real-valued operations, obtained through the summation of all propagation operations (1,704,648 + 1,706,376 + 11,740) and modulation operations (8,712 + 3,528). The MN-DNN exhibits an overall latency of approximately 53.3 ns, as detailed in Table R9.\nTable R9. Time delay of the MN-DNN system\nEvents Nonlinear layers (3 layers) Linear layers (3 layers) Propagation\nTime delay 53.1ns 100.8 ps 2.33 ns\nBased on the aforementioned total operation count of 3,435,004 real-number operations and the measured total latency of 53.3 ns, the computational throughput of the system is calculated to be 64.447 TOPS/s. Under a total system power consumption of 252.352 W, the energy efficiency reaches 0.25TOPS/W.\nComputing Speed\nEnergy efficiency = = 0.25 TOPS/W (R1)\nPower Consumption\nNotably, the proposed network architecture exhibits excellent scalability characteristics, with the energy efficiency improving proportionally to network scale. When increasing the single nonlinear layer dimensionality from 14×14 to 56×56 (a 16× area scaling factor), the corresponding linear layer scales commensurately from 22×22 to 88×88. This architectural expansion achieves a 256-fold enhancement in computational throughput while maintaining a favorable power scaling relationship, with total power consumption increasing by only 16 times. The resulting improvement in energy efficiency reaches approximately 4 TOPS/W, representing a 16-fold gain over the baseline configuration. This scaling behavior demonstrates the architecture's particular suitability for energy-constrained large-scale neural network implementations.\n2) Comparisons with other DNN networks\nRegarding total power consumption, our design achieves a power consumption of 252.25W, which is comparable to other 3D reconfigurable diffractive neural network implementations (typically consume hundreds of watts), as shown in Table R10. The system achieves a power efficiency of 0.25 TOPs/W, which is competitive with existing approaches.\nTable R10. Comparative analysis of power consumption and energy efficiency with DNN architectures\nSystem\nPower\nEnergy\nComputing Speed Architecture Integrated\nConsumption\nRepresentative works\n(TOPs/s) Type Nonlinearity1 Efficiency2\n(W)\n(TOPs/W)\nNat Photonics 15, 367-373\n159.4 114.1 3D No 0.71\n(2021).\n[R17]\nNat Electronics 5, 113-122\n~100 79.64 3D No 0.7964\n(2022). China [R18]\nNat Commun 15, 10692\n0.14 0.0017 3D Yes 0.011\n(2024). [R19]\nNat Electron 6, 319-328\n\\ 0.0032 Planar No 0.00001667\n(2023). [R20]\nMN-DNN\n252.352 64.44 3D Yes 0.25\n(this work)\n1. Integrated nonlinearity refers to activation functions implemented entirely within a single processing unit using integrated components, where the nonlinear transformation is achieved through intrinsic device characteristics.\nNon-integrated nonlinearity describes activation functions that require additional, separate hardware components or computational resources to perform the nonlinear transformation.\n2.System energy efficiency is calculated based on the total energy consumption of the system for the architectures, including the nonlinear energy consumption of every active device.\nFurther power reduction can be attained by integrating ultra-low-power components such as the MAX2659 RF amplifier (power consumption: 6.56 mW), LT5537 RF detector (40.5 mW), and LPV821 op-amp (2.844 μW), potentially reducing total power consumption by~90%. Notably, our architecture exhibits strong scaling advantages. When scaled to 56×56 nonlinear units per layer (with linear layers expanding to 88×88), energy efficiency improves to 4 Tops/W. This scalability makes our approach particularly compelling for large-scale neural network deployment where power efficiency is critical.\nReferences:\n[R17] Zhou T, et al. Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit. Nat Photonics 15, 367-373 (2021).\n[R18] Liu C, et al. A programmable diffractive deep neural network based on a digital-coding metasurface array. Nat Electron 5, 113–122 (2022).\n[R19] Song A, Murty Kottapalli SN, Goyal R, Schölkopf B, Fischer P. Low-power scalable multilayer optoelectronic neural networks enabled with incoherent light. Nat Commun 15, (2024).\n[R20] Gao X, et al. Programmable surface plasmonic neural networks for microwave detection and processing. Nat Electron 6, 319-328 (2023).\nAction Taken:\n[Architecture of nonlinear metasurface unit]\nEach nonlinear unit exhibits a total power dissipation of approximately 429mW, which consists of 150 mW for the RF amplifier,53mW for the RF detector, and 224mW for the voltage adder.\n[Handwritten digits classification by MN-DNN]\nThe total power of the MN-DNN is about 252.352W (details see Supplementary Note 16).\n[Discussion]\nFuture improvements in energy efficiency can be achieved by adopting faster and lower-power devices while scaling up the network. Due to its strong scalability, our network achieves higher power efficiency with larger single-layer configurations. For example, the system with a 56×56-unit single nonlinear layer reaches 4 TOPS/W, positioning it among the state-of-the-art in energy-efficient designs.\n[Supplementary Note 16: Power consumption of MN-DNN]\nThe aforementioned analyses have been included [Supplementary Note 16: Power consumption of MN-DNN].\nComment 1.7:\nCan the authors explain the digital training algorithm a bit more? Have they modeled the behavior of the RF components in the training software?\nResponse:\nWe sincerely appreciate your valuable suggestion. As you rightly pointed out, these implementation details are essential for reproducibility. Accordingly, we have added comprehensive training specifications in the revised manuscript, encompassing: input preprocessing, optimization algorithm, learning rate scheduling, number of training epochs, and other relevant hyperparameters.\nDigital training algorithm\nFigure R16. Digital training flowchart of the nonlinear system. a, Schematic illustration of nonlinear network training, in which all parameters are updated jointly. b, Equivalent structural block diagram of the nonlinear network. The complete details appear as follows: First, the MNIST dataset is down sampled and binarized (details see Supplementary Note 5). Subsequently, the power and phase distribution generated by the horn antenna excitation at the input plane is multiplied by the processed MNIST dataset (Supplementary Note 10), which serves as the input to the MN-DNN. The nonlinear network consists of 1,458 trainable parameters, with 484 parameters per linear layer and 2 parameters (threshold and slope) per nonlinear layer, yielding a total of 484 × 3 + 2 × 3 = 1,458. During training, the input data is first propagated forward through the network to generate predictions. The model’s performance is evaluated using a weighted composite loss function, combining categorical cross-entropy loss (60%) and mean squared error loss (40%) to balance classification and regression objectives. The total loss is then minimized using the Adam optimizer, initialized with a learning rate of 0.01. By leveraging adaptive moment estimation, the optimizer dynamically adjusts parameter updates based on the first and second moments of the gradients. To enhance convergence, an exponential learning rate decay strategy is employed, where the learning rate decays to 96% of its current value every 100,000 training steps. This approach facilitates rapid initial optimization while enabling finer adjustments in later training phases. Throughout the iterative process, classification accuracy is monitored to assess the alignment between predictions and ground truth labels, providing real-time feedback on model performance. A total of 20 training epochs were conducted in Python.\nRF components math modeling\nFigure R17. Mathematical model of the nonlinear layer with training optimization and hardware implementation a, The measured RF device characteristics are fitted into a ReLU-like nonlinear function using scatter data. b, The nonlinear function is expressed as f(x) = b × max (0, x−a), where a represents the reconfigurable threshold and b denotes the reconfigurable slope. c, Through training, optimal threshold and slope parameters are obtained. d, The trained parameters are mapped to voltage configurations by establishing relationships between Vbias and threshold a, and between VCC and slope b, thereby completing the training-to-hardware configuration loop.\nThe nonlinear characteristics achieved through the combined effects of the RF components are experimentally characterized and mathematically modeled within the training software. In detail, we systematically measure the transmission coefficients across varying incident power levels, with comprehensive testing methodologies detailed in Supplementary Note 2. Based on the measured data, we construct scatter plots and employ curve fitting to derive the mathematical expression of the activation function, which exhibits a ReLU-like form:\n𝑦 = 𝑏×max (0 ,𝑥 −𝑎) (S1)\nwhere x represents the input power, y denotes the output power, a is the threshold parameter, and b is the slope parameter (as shown in Fig.R17b). Notably, the key characteristic parameters of this activation function (the threshold a and slope b) can be dynamically tuned by adjusting the control voltages V and V , respectively. As illustrated in Fig.R17b, V enables continuous modulation CC BIAS CC of the slope b over a range of 0 to 8 within an operating voltage range of 0 to 3 V, while V precisely BIAS controls the threshold a within a range of 0 to 15 mW for bias voltages ranging from −0.7 to 0.3 V.\nThe optimization procedure employs parameter ranges of 0~8 for the nonlinear layer's slope b and 0~15 for its threshold a.\nAfter extensive training, we identify the optimal combinations of threshold and slope parameters for each of the three nonlinear layers (Fig.R17c). These trained parameters are precisely mapped to the corresponding applied voltages using pre-established voltage-parameter relationships (the V -slope CC and V -threshold curves, as shown in Fig.R17d). Finally, by applying the V and V voltages BIAS CC BIAS to the DC sources, we achieved hardware configuration of the MN-DNN nonlinear layer, completing the full closed loop from theoretical model to physical implementation.\nAction Taken:\n[Architecture of nonlinear metasurface unit]\nFig. 2l indicates that higher V reduces the nonlinear activation function’s threshold, enabling BIAS variable nonlinear activation thresholds by modulating the bias voltage V . Fig. 2m shows that BIAS increasing V steepens the slope of the nonlinear activation function, resulting in elevated amplifier CC gain and transmission coefficients for the metasurface units. Based on the measured data, we construct scatter plots and employ curve fitting to derive the mathematical expression of the activation function, which exhibits a ReLU-like form:\n𝑦 = 𝑏×max (0 ,𝑥 −𝑎) (1)\nwhere x represents the input power, y denotes the output power, a is the threshold parameter, and b is the slope parameter.\n[Handwritten digits classification by MN-DNN]\nThe linear network comprises three linear phase-modulation layers spaced 20 cm apart, each composed of a 22×22 array of 19 mm × 19 mm unit cells. The phase of each unit serves as a trainable parameter, adjustable within the range of 0 to 2π (details in Supplementary Note 5). The nonlinear network features an alternating linear-nonlinear structure with three linear layers interspersed with three nonlinear metasurface layers, spaced at 10 cm intervals. Each nonlinear layer contains two independent trainable parameters: the threshold and slope of its ReLU activation function, with layer-specific values (RF components math modeling see Supplementary Note 8). The nonlinear network consists of 1,458 trainable parameters, with 484 parameters per linear layer and 2 parameters (threshold and slope) per nonlinear layer, yielding a total of 484 × 3 + 2 × 3 = 1,458. …. The loss function is a weighted sum of the Mean Squared Error and SoftMax Cross-Entropy, with coefficients 0.4 and 0.6, respectively. The gradient descent optimization is performed using the Adam algorithm.\n[Supplementary Note 8: RF components math modeling]\nThe aforementioned results and analyses have been included in Supplementary Note 8.\nComment 1.8:\nIn Fig. 2 the word “backward” is a bit confusing. One may think that signal propagates backward, too. Transmit and receive antenna might be a better choice.\nResponse:\nWe sincerely appreciate your constructive feedback. In accordance with your suggestion and to enhance clarity, we have replaced forward structure and backward structure with receive antenna and transmit antenna, respectively.\nAction Taken:\n[Architecture of nonlinear metasurface unit]\nFig. 2 | Nonlinear metasurface unit configuration. a The nonlinear metasurface unit, featuring four metal and three dielectric layers, with F4B as the top and bottom dielectrics and FR4 as the middle layer. The unit functions as a summation, nonlinear activation, and amplitude modulation device. b Front and back views of the nonlinear metasurface unit, featuring an octagonal receive antenna on the front with an integrated RF detector, and on the back, a corresponding octagonal transmit antenna accompanied by an RF amplifier and voltage adder. c Operational diagram of the nonlinear metasurface unit, showing the receive antenna directing EM energy to the back for amplification and a portion to the RF detector for rectification. The rectified voltage and an external bias voltage can modulate the RF amplifier’s DC control voltage V , with V controlling the gain. The amplified RF energy CTRL CC is then radiated by the transmit antenna. d The output DC voltage of the RF detector linearly scales with the incident power density on the nonlinear metasurface. e The gain of the RF amplifier is jointly determined by DC voltages V and V . f The RF amplifier’s gain exhibits a minimal value at low V , effectively cuts off, and subsequently CC CTRL CC increases to a saturation point with further V increments. g The RF amplifier’s gain is minimal at low V , CC CTRL effectively cuts off, and then rises to a plateau as the additional V increases. h For V > 0.7V, the nonlinear CTRL BIAS metasurface unit is operated linearly, with the output energy directly proportional to the input energy, and gain escalating with V . i The transmission coefficient of the nonlinear metasurface unit rises from a minimal value at CC low incident intensities to a plateau at higher levels, with V reducing the energy threshold for transition from BIAS the cutoff to transmission. j The transmission coefficient of the nonlinear metasurface unit escalates with the incident energy, reaching a plateau, and is enhanced by increased V at constant input energy levels. k With V = -0.15V CC BIAS and V = 2V, the nonlinear metasurface unit exhibits zero transmission for the incident energies below 1 mW and CC a linear increase above this threshold, akin to a ReLU function. l The unit’s nonlinear activation threshold is inversely related to V . m The unit’s gain is directly proportional to V . BIAS CC\nThe RF energy is coupled by the receiving antenna, passes through the active circuit, and is radiated by the transmitting patch antenna into the air, as illustrated in Fig. 2a. The front-side passive structure comprises an octagonal receiving patch antenna and a T-junction unequal power divider.…. The unit’s back-side features an octagonal transmitting patch antenna identical in size to the receiving antenna, as illustrated in Fig. 2b. The operational diagram of the nonlinear metasurface unit is shown in Fig. 2c. The spatial RF energy, post-coupling by the receiving antenna and microstrip line transmission, is split by the T-junction power divider. The bulk of energy is amplified by the RF amplifier and proceeds to the transmitting antenna, while a fraction of the energy is diverted to a high- speed RF detector and is converted into direct current (DC) voltage V .\nOUT\nComment 1.9:\nAuthors mention (line 118) that the nonlinearity in neural networks is in milliseconds and microseconds range. What kind of neural network are they referring to? If it is optical, then much faster nonlinearities have been demonstrated. Please clarify.\nResponse:\nWe gratefully acknowledge the reviewers’ insightful comment. Concerning the discussion of nonlinear responses at millisecond and microsecond timescales in line 118, we clarify that this refers to conventional nonlinear neural network systems based on photoelectric conversion [R21]-[R23] and MCU/ADC-DAC architectures [R24]. We fully agree with the reviewers’ observation that the field of optical nonlinearity has achieved remarkable progress toward faster timescales. Indeed, cutting-edge techniques such as electro-optic self-modulation [R25], semiconductor carrier modulation [R26], and quantum dot thin-film [R27] approaches have demonstrated ultrafast nonlinear responses at picosecond timescales, significantly advancing the temporal performance of diffractive neural networks.\nIt should be noted that our study focuses on a nonlinear system operating in the microwave regime, which offers distinct practical advantages. Microwave-based deep neural networks can be seamlessly integrated into real-world applications, including wireless communications, remote sensing, radar imaging, and security screening systems. This analog-domain processing paradigm effectively mitigates the limitations of conventional digital baseband processors, such as excessive hardware complexity and high-power consumption.\nMoreover, while most state-of-the-art ultrafast nonlinear systems rely on planar optical computing, our proposed spatial diffractive neural network exhibits two key differentiating features: (1) full three-dimensional spatial processing capability, enabling direct manipulation of free-space electromagnetic waves beyond the constraints of planar architectures, and (2) superior spatial scalability, facilitating the implementation of large-scale trainable nodes. These unique characteristics position our microwave-based system as a complementary approach to existing optical solutions, with promising potential for practical deployment.\nReferences:\n[R21] Song A, Murty Kottapalli SN, Goyal R, Schölkopf B, Fischer P. Low-power scalable multilayer optoelectronic neural networks enabled with incoherent light. Nat Commun 15, (2024).\n[R22] Zhou T, et al. Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit. Nat Photonics 15, 367-373 (2021)..\n[R23] Wang T, et al. Image sensing with multilayer nonlinear optical neural networks. Nat Photonics 17, 408-415 (2023).\n[R24] Gao X, et al. Programmable surface plasmonic neural networks for microwave detection and processing. Nat Electron 6, 319-328 (2023).\n[R25] Bandyopadhyay S, et al. Single-chip photonic deep neural network with forward-only training. Nat Photonics 18, 1335-1343 (2024).\n[R26] Wu T, Li Y, Ge L, Feng L. Field-programmable photonic nonlinearity. Nat Photonics, (2025).\n[R27] Huang Z, Shi W, Wu S, Wang Y, Yang S, Chen H. Pre-sensor computing with compact multilayer optical neural network. Sci Adv 10, (2024).\nAction Taken:\n[Introduction]\nLow latency is essential for swift information processing, yet the nonlinearities in current neural networks often introduce millisecond to microsecond delays owing to their reliant on photoelectric conversion2, 4, 6 or MCU/ADC-DAC architectures5, hindering the potential for computation at the speed of light.\nComment 1.10:\nLine 174, what does the word “atom” refer too? Seems to be a typo.\nResponse:\nThank you very much for your reminder. We have changed \"atom\" to \"unit\".\nAction Taken:\n[Architecture of nonlinear metasurface unit]\nThe details of the nonlinear unit are provided in Supplementary Note 1.\nREVIEWER #2\nWe sincerely appreciate the reviewer for the time and effort invested in reviewing our manuscript and offering a wealth of insightful comments and thoughtful suggestions. These comments and suggestions are extremely valuable to enhance the quality of our manuscript. Following your constructive guidance, we carefully revised the paper to improve its clarity and comprehensibility. Below, we provide our one-by-one responses (in black fonts) to the referees’ comments (in blue fonts).\nRemarks to the Author:\nIn \"Multilayer nonlinear diffraction neural networks with programmable and fast ReLU activation function\", the authors propose a microwave-domain multilayer nonlinear diffractive neural network (MN-DNN) that integrates programmable metasurfaces with RF detectors, amplifiers, and voltage adders to realize a fast, programmable ReLU-like activation with nanosecond latency. Each metasurface layer performs linear transformations, while custom electronic circuits implement the nonlinear activations by detecting, filtering, and re-amplifying signals. This hybrid design enables fast and low-power processing with tunable thresholds and gain, supporting other activations like Tanh and Softmax through simple circuit modifications. The system achieves 92.83% accuracy on MNIST, outperforms linear models by over 4% on both MNIST and Fashion-MNIST, and classifies human postures with 93.06% accuracy. Its compact, reconfigurable architecture offers low latency and high energy efficiency, addressing key limitations of prior diffractive systems and highlighting the potential of combining optics-inspired principles with active electronics for fast, efficient neural processing.\nResponse:\nThank you very much for your kind support and encouraging comments. We have carefully addressed your questions point by point. Please refer to the details as below.\nComment 2.1:\nOn page 3, line 81, you state that the MN-DNN model demonstrates an enhancement in image classification performance on the MNIST and Fashion-MNIST datasets, achieving an accuracy of 92.83% on MNIST. However, the classification accuracy for the Fashion-MNIST dataset is not mentioned until much later, as far as I can see. To improve clarity and provide a more comprehensive introduction, it would be beneficial to also include the classification accuracy for Fashion-MNIST early on in the paper, to give the readers a quantitative anchor and also to be able compare the result with literature.\nResponse:\nWe sincerely appreciate your constructive suggestion. As suggested, we have added the accuracy results of Fashion-MNIST to the relevant section in the Introduction to enhance clarity for readers.\nAction Taken：\n[Introduction]\nWe show that the MN-DNN can enhance the image classification on Modified National Institute of Standards and Technology (MNIST) and Fashion-MNIST datasets, with the accuracy of 92.83% on MNIST and 78.8% on Fashion-MNIST, surpassing the linear networks by over 4% in both cases.\nComment 2.2:\nIt is obvious that the nonlinear results in Fig 3 are surpassing the linear results, as this is already expected from this type of a classification task. However, all the examples shown here are in Fig 3f belong to cases where the digits are misclassified by the linear network. For comparison purposes, I think it would be more valuable to demonstrate also one example where the linear network can also perform well. Otherwise, it seems like the linear network cannot classify any of the images, which is clearly incorrect since the classification accuracy reported is 88% in Fig 3g. The same can also be said about the fashion MNIST classification in Fig 4. Here, it seems like the nonlinear and linear networks have returned different results for all inputs shown, which clearly cannot always be the case with over 70% accuracy.\nResponse and Action Taken:\nThank you very much for your valuable comment. We revised Fig.3 and Fig.4 by adding the correctly identified components (for both linear and nonlinear networks) along with corresponding explanations to enhance clarity and avoid potential misunderstandings. Additionally, we have included additional experimental data in the Supplementary Note 14 and Supplementary Note 18, demonstrating accurate identification for both linear and nonlinear networks, enhancing the completeness of results.\n[Handwritten digits classification by MN-DNN]\nTo further analyze the networks' behavior, we perform output field distribution scans on a subset of images. For simple images with distinct features, such as the handwritten digit \"7\" (Fig. 3g), both linear and nonlinear networks achieve accurate recognition, with their output energies precisely concentrated on the target category's corresponding region (additional results in Supplementary Note 14). However, when processing complex images with easily confusable features, the nonlinear network exhibits a pronounced advantage. As shown in Fig. 3h, for a \"4\" sample prone to confusion with \"9\", the nonlinear network's output energy correctly localized to the \"4\" region, whereas the linear network misclassified it, with its peak energy erroneously appeared in the \"9\" region (despite the \"4\" region having the second-highest energy). This phenomenon remained consistent across other complex image classifications (further comparisons in Supplementary Note 14), demonstrating the nonlinear network's superior performance in handling complex images.\nFig. 3 | Handwritten digits classification. a Illustration of the MNIST dataset for recognition, including 10 classes of handwritten digits (0-9). b Schematic of the input image encoding process, where binarized images are translated into transmission coefficients of the input layer metasurface. c Photo of the testing scenario, where a horn antenna illuminates the input layer, directing the EM waves carrying the image information into MN-DNN. The output field distribution is scanned using a two-dimensional near-field scanner. d Comparison of linear and nonlinear networks is facilitated by two optimized designs: a nonlinear network with interleaved 3 linear and 3 nonlinear layers, and a linear network with only 3 linear layers. e t-SNE visualization of test set outputs for both networks, demonstrating that the nonlinear network achieves more distinct class separation and tighter intra-class clustering. f Confusion matrix for 5000 test images, showing the 92.4% and 88.5% accuracy of the nonlinear network and linear network. g Sampled output field distributions generated by the linear and nonlinear networks for one simple image, demonstrating that both networks successfully focus on the target region. h Example comparative output field distributions for two complex images from both networks, with the nonlinear network focusing on correct regions and the linear network on incorrect regions.\n[Fashion-MNIST classification by MN-DNN]\nFig. 4 | Fashion-MNIST dataset recognition. a Overview of 10-class fashion item dataset for recognition. b Comparison of two network architectures: the nonlinear network with alternating linear and nonlinear layers, and the linear network with only linear layers. c Example output field distributions generated by linear and nonlinear networks for two simple images, demonstrating that both networks successfully focus on the target regions. d Sampled output-field distributions for three complex images from both networks, with the nonlinear network focusing on correct regions and the linear network on incorrect regions. e The recognition accuracy increases with the number of nonlinear layers in the network. f The effect of nonlinear-layer positioning on the accuracy, with the front and back placements outperforming the central insertions. g The impact of activation function on the recognition accuracy, with ReLU showing the highest improvement among the functions tested.\nFor simple and distinct images like the ankle boot in Fig.4c, both linear and nonlinear networks achieve accurate classification, with output energies correctly focused on the target region (more results see Supplementary Note 18). However, when dealing with complex images characterized by similar and confusable features, the nonlinear network demonstrates a clear advantage. As illustrated in Fig.4d, the nonlinear network correctly classifies the sneaker image (often confused with ankle boots), with output energy concentrated specifically in the sneaker region. In contrast, the linear network misclassifies this input, displaying peak energy in the wrong ankle boot region despite secondary activation in the correct sneaker category. This consistent pattern across challenging images (more results see Supplementary Note 18) confirms nonlinear networks' superior performance in complex images recognition tasks.\n[Supplementary Note 14: Linear and nonlinear network performance on MNIST]\nFor simple images, both linear and nonlinear networks achieve accurate recognition by focusing energy at the correct positions on the output plane. Taking the first image labeled \"0\" in Supplementary Figure S24 as an example, both linear and nonlinear neural networks successfully concentrate the output field energy in the region corresponding to the digit \"0,\" thereby achieving correct classification. A similar behavior is observed for the remaining images in Fig. S24.\nSupplementary Figure S24 | Performance comparison of linear and nonlinear networks on simple MNIST images. For simple, well-defined images in the MNIST dataset, both linear and nonlinear diffractive neural networks produce output fields focused at the correct positions, enabling accurate classification.\n[Supplementary Note 18: Linear and nonlinear network performance on Fashion-MNIST]\nFor simple images, both linear and nonlinear networks achieve accurate recognition by focusing energy at the correct positions on the output plane. Taking the first image labeled ankle boot in Figure S1 as an example, both linear and nonlinear neural networks successfully concentrate the output field energy in the region corresponding to the ankle boot, thereby achieving correct classification. A similar behavior is observed for the remaining images in Fig. S32.\nSupplementary Figure S32 | Performance comparison of linear and nonlinear networks on simple, well- defined Fashion_MNIST images. For simple, well-defined images in the Fashion_MNIST dataset, both linear and nonlinear diffractive neural networks produce output fields focused at the correct positions, enabling accurate classification.\nComment 2.3:\nFrom a similar perspective, the nonlinear network has significantly more layers and potentially more trainable parameters, which depends on how nonlinearity was trained. Could the improved performance be partially attributed to the increased network depth rather than solely the nonlinearity? A comparison with a 6-layer linear network, or one that has an equal number of trainable parameters would be more informative. The same problem also exists for the analysis in Supplementary Figure S12 and S13. Showing that additional nonlinear layers increase accuracy does not constitute a novel finding on its own, as this is already well-known and expected.\nResponse:\nWe sincerely appreciate the reviewer’s insightful comments regarding the origin of performance improvements in nonlinear networks. To address whether these gains stem from increased network depth or intrinsic nonlinear properties, we perform comprehensive analyses and additional controlled experiments.\nA key feature of our architecture is that each nonlinear layer employs globally shared parameters, specifically, all 14×14 units within a nonlinear layer utilize identical threshold and slope values. Thus, adding a nonlinear layer introduces only two trainable parameters (threshold and slope), making its contribution to the total parameter count negligible.\nTo rigorously decouple the effects of depth from nonlinearity, we design a matched six-layer linear control network with identical parameterization. Specifically, we compared 3 networks:\n1) A baseline three-layer linear network (22×22 independently tunable units per layer, 1,452 total parameters);\n2) A network comprising three linear layers interleaved with three nonlinear layers, with the nonlinear layers contributing only six extra parameters (total parameters: 1,458);\n3) A novel six-layer linear network where the additional three layers apply globally consistent linear modulation (two parameters per layer, aptitude and phase, six in total), maintaining both the total parameter count (1,458) and network depth identical to the nonlinear net.\nFigure R18 | Network architectures and parameter comparisons. a, A baseline three-layer linear network (22× 22 independently tunable units per layer, 1,452 total parameters). b, A network comprising three linear layers interleaved with three nonlinear layers, with the nonlinear layers contributing only six extra parameters (total parameters: 1,458). c, A novel six-layer linear network where the additional three layers apply globally consistent linear modulation (two parameters per layer, six in total), maintaining both the total parameter count (1,458) and network depth identical to the nonlinear net.\nFigure R19 | Recognition accuracy of three network architectures on the MNIST dataset. a. A three-layer linear network (trainable parameters: 1,452) achieves a recognition accuracy of 0.8863. b. A novel six-layer linear network with globally consistent linear modulation in the additional three layers (trainable parameters: 1,458) attains an accuracy of 0.889. c. A hybrid network consisting of three linear layers alternated with three nonlinear layers (trainable parameters: 1,458) reaches an accuracy of 0.9281. d. Performance comparison of the three network architectures.\nFigure R20 | Recognition accuracy of three network architectures on the Fashion_MNIST dataset. a. A three- layer linear network (trainable parameters: 1,452) achieves a recognition accuracy of 0.7443. b. A novel six-layer linear network with globally consistent linear modulation in the additional three layers (trainable parameters: 1,458) attains an accuracy of 0.7508. c. A hybrid network consisting of three linear layers alternated with three nonlinear layers (trainable parameters: 1,458) reaches an accuracy of 0.788. d. Performance comparison of the three network architectures.\nWe systematically evaluate the performance of three network architectures using the MNIST and Fashion-MNIST datasets as benchmark tasks. For MNIST classification, as shown in Fig.R19, the baseline three-layer linear network achieves an accuracy of 88.63%. Extending this to a six-layer linear architecture, where each additional layer introduces only two global modulation parameters (totaling six additional parameters), yields a marginal improvement to 88.9%, representing a gain of less than 0.3%. In contrast, introducing three nonlinear layers (with identical parameter) substantially improves accuracy to 92.81%, corresponding to a 4.18% enhancement. These results demonstrate that performance gains arise primarily from nonlinear transformations rather than increased depth or minor parameter adjustments. A similar pattern emerges for Fashion-MNIST (as shown in Fig.R20).\nThe three-layer linear baseline achieves 77.43% accuracy, while expanding to six linear layers leads to slight degradation (75.08%), reinforcing that simply adding linear depth fails to improve performance. However, incorporating nonlinear layers boosts accuracy to 78.8%, outperforming purely linear architectures despite equivalent parameter counts.\nThe systematic comparisons reveal that performance improvements depend critically on nonlinear transformations rather than network depth or parameter increments. These findings underscore the effectiveness of our approach in enhancing network expressivity through targeted nonlinearity incorporation.\nRegarding to Supplementary Figures S12 and S13, while the phenomenon that adding nonlinear layers can improve recognition accuracy is indeed well-established in literature, the key innovation of our work lies in the in-depth analysis of the unique role played by nonlinear metasurfaces in diffractive neural networks from a hardware implementation perspective. These experimental results not only reconfirm the general effectiveness of nonlinear layers, but more importantly, they provide empirical validation for our proposed nonlinear metasurface-based hardware architecture.\nAction Taken:\n[Handwritten digits classification by MN-DNN]\nThe training outcomes indicate that the nonlinear network achieves a recognition accuracy of 92.81%, which is 4.18% improvement over the 88.63% accuracy of the linear network. Moreover, a linear network with the same number of trainable parameters as nonlinear network only reaches 89% accuracy, verifying that the performance improvement is attributed to the nonlinear activation, as presented in Supplementary Note 11.\n[Fashion-MNIST classification by MN-DNN]\nThe experimental results demonstrate that the nonlinear diffractive network achieves 78.8% classification accuracy on Fashion-MNIST, outperforming the baseline linear network (74.43%) by 4.37%. The accuracy comparison with other networks is given in Supplementary Note 17. Crucially, when using a linear network with the same number of trainable parameters as the nonlinear network, the accuracy only improves marginally to 75.08%, as illustrated in Supplementary Note 11, which conclusively verifies that the performance enhancement stems from the introduced nonlinearity.\n[Supplementary Note 11 Comparative Analysis with Linear Networks of Equivalent Trainable Parameters]\nThe aforementioned results and analyses have been included in Supplementary Note 11.\nComment 2.4:\nIn the dataset-based recognition tasks for both MNIST and Fashion-MNIST, as outlined in Supplementary Note 4, you describe a process in which the original 28×28-pixel grayscale images are first down sampled to 14×14 pixels using interpolation. Following this, a binarization is applied, where pixel values exceeding a threshold of 0.15 are set to 1. While this down sampling and binarization process is a standard approach in some image processing tasks, it is important to note that the binarization step can have a significant impact on dataset difficulty and model performance. By reducing pixel precision, one inevitably loses valuable information, which could directly contribute to a decrease in recognition accuracy. It would be beneficial to discuss the potential effects of this binarization on the presented results, as understanding its impact is crucial for readers evaluating the robustness of this approach.\nResponse:\nWe sincerely appreciate the reviewers’ valuable comments. We fully agree that the binarization of input images would impact model performance. To systematically evaluate the effects of down sampling and binarization on recognition performance, we conducted comparative experiments on the MNIST and Fashion-MNIST datasets. These experiments employed three types of input: original images, down sampled images, and down sampled and binarized images, tested separately in purely linear and nonlinear networks.\nThe results reveal distinct performance trends. For Fashion-MNIST, the linear network achieves accuracies of 78.00% (original images), 76.98% (down sampled images), and 74.43% (down sampled + binarized images) for the three input types, as shown in Fig. R21. While the nonlinear network attains 83.71%, 82.68%, and 78.80% respectively Fig.R22. Similarly, on MNIST, the linear network achieves 91.28%, 91.07%, and 88.63% (Fig.R23), whereas the nonlinear network reaches 94.78%, 94.68%, and 92.81% (Fig.R24). These findings demonstrate that while down sampling has a relatively minor effect on accuracy, binarization leads to a more substantial performance decline of 2–5%, consistent with the reviewers’ observation that binarization discards valuable image information.\nWe adopt binarization primarily for two reasons. First, our study focuses on comparing the relative performance of nonlinear versus linear networks under identical input conditions, making input consistency more critical than absolute accuracy. Second, binarization significantly reduces control circuit complexity, a key consideration for our hardware implementation.\nWe acknowledge the limitations of this down sampling and binarization process and explicitly discuss the performance trade-offs in the revised manuscript. To address these limitations, we propose future enhancements, including the integration of a DAC module to enable approximately continuous grayscale input processing. This modification, compatible with our current input layer architecture, is expected to preserve richer image details and improve recognition accuracy while retaining circuit simplicity. This refinement will better balance performance and implementation complexity. More details and analyses are available in our response to Comment 1.4.\nFigure R21 | Impact of three types of Fashion_MNIST dataset inputs on the recognition accuracy of a linear network. a, Input images are the original Fashion_MNIST images, achieving a recognition accuracy of 78.48%. b, Input images are down sampled to 14×14 pixels, yielding a recognition accuracy of 76.95%. c, Input images are down sampled to 14×14 pixels and then binarized with a threshold of 0, resulting in a recognition accuracy of 74.43%.\nFigure R22 | Impact of three types of Fashion_MNIST dataset inputs on the recognition accuracy of a nonlinear network. a, Input images are the original Fashion_MNIST images, achieving a recognition accuracy of 83.71%. b, Input images are down sampled to 14×14 pixels, yielding a recognition accuracy of 82.68%. c, Input images are down sampled to 14×14 pixels and then binarized with a threshold of 0, resulting in a recognition accuracy of 78.80%.\nFigure R23 | Impact of three types of MNIST dataset inputs on the recognition accuracy of a linear network. a, Input images are the original MNIST images, achieving a recognition accuracy of 91.28%. b, Input images are down sampled to 14×14 pixels, yielding a recognition accuracy of 91.07%. c, Input images are down sampled to 14×14 pixels and then binarized with a threshold of 0, resulting in a recognition accuracy of 88.63%.\nFigure R24 | Impact of three types of MNIST dataset inputs on the recognition accuracy of a nonlinear network. a, Input images are the original MNIST images, achieving a recognition accuracy of 94.78%. b, Input images are down sampled to 14×14 pixels, yielding a recognition accuracy of 94.68%. c, Input images are down sampled to 14×14 pixels and then binarized with a threshold of 0, resulting in a recognition accuracy of 92.81%.\nAction Taken：\n[Supplementary Note 27: Effect of image preprocessing on recognition accuracy]\nThe aforementioned results and analyses have been included in Supplementary Note 6: Effect of image preprocessing on recognition accuracy].\n[Discussion]\nFor image classification tasks, future architectural improvements could target two key aspects to enhance recognition accuracy: Firstly, integration of a digital-to-analog converter (DAC) with the input-layer metasurface to enable continuous grayscale inputs, thereby improving information fidelity and subsequent feature extraction accuracy (Supplementary Note 30).\nComment 2.5:\nThe binarization process inevitably leads to a decrease in performance, even when employing fully digital neural networks. Specifically, when this dataset is used for classification in digital neural networks, the accuracy values observed in the literature are typically lower. Notably, in the case of Fashion-MNIST, you observed a maximum accuracy of only 77% when utilizing ReLU activation, which is substantially lower than the results achieved in the original dataset for digital classification. Can this discrepancy be attributed to the down sampling and binarization techniques applied? How can one explain the significant drop in efficiency compared to typical digital models that achieve over 90% on the same task?\nIn other words, can the observed decrease in accuracy be more a result of the dataset transformation, rather than a limitation of the MN-DNN model itself? To further investigate this, I recommend training a traditional digital neural network (such as a CNN or FCNN) with an equivalent number of trainable parameters to those in MN-DNN, using the down sampled and binarized MNIST and Fashion-MNIST datasets. By comparing the performance of this digital network against the results from MN-DNN, the supplementary material could offer valuable insights into the specific impact of the dataset preprocessing on the performance and highlight any differences between MN-DNN and standard digital neural networks.\nResponse:\nWe sincerely appreciate the reviewers’ valuable comments. Regarding your concern about the impact of binarization on model performance, we conduct an in-depth comparative study. Through comprehensive simulation and comparative analysis, we confirm that the reduction in classification accuracy on the Fashion-MNIST dataset is predominantly attributable to the dataset transformation (the downsampling and binarization operations) rather than any fundamental constraints of the MN-DNN model.\nWe would analyze the MN-DNN’s accuracy from two key aspects:\n(1) Performance comparison on traditional digital neural networks\n(2) Performance comparison on other nonlinear DNNs\nPerformance comparison on traditional digital neural networks\nTo systematically evaluate the respective effects of dataset preprocessing and model architecture, we carefully design a fully connected network (FCN) with comparable parameters to MN-DNN, as well as a more representative CNN model as benchmarks. Specifically, the FCN adopts a three-layer structure of 196-7-10, totaling 1,459 parameters, which is highly comparable to MN-DNN’s 1,458 parameters. Additionally, we construct a CNN architecture (approximately 106,000 parameters) comprising convolutional layers, pooling layers, and fully connected layers to investigate the effects of deeper feature extraction. Both models use the same 14×14 binarized input images as MN-DNN.\nThe experimental results demonstrate that on the binarized MNIST dataset, the FCN, CNN, and MN-DNN achieve accuracies of 85.97% (FCN, Fig. R25), 93.57% (CNN, Fig.R26), and 92.83% (MN-DNN), respectively. On the more challenging Fashion-MNIST dataset, the accuracies are 75.46% (FCN, Fig.R27), 83.22% (CNN, Fig.R28), and 78.8%(MN-DNN), respectively (see Table R11 for the comparative results). MN-DNN performs comparably to or even better than the nonlinear FCN, while being lower but still competitive with the CNN.\nRemarkably, MN-DNN achieves classification performance close to that of traditional digital neural networks, which strongly validates the effectiveness of its architecture. We agree that the performance decline primarily stems from information loss due to the binarization process, rather than inherent limitations in MN-DNN’s design.\nFigure R25 Training of the binarized MNIST dataset on a fully connected network (FCN). a, The processed MNIST dataset is trained on an FCN with the following architecture: a 14×14 input layer (representing the downsampled and binarized MNIST dataset), a fully connected layer with 7 nodes, a ReLU nonlinear activation layer, a fully connected layer with 10 nodes, a Softmax nonlinear activation layer, and a 10-class classification output. b, The confusion matrix of the trained network, achieving an identification accuracy of 85.97%. c, The identification accuracy during the training process. d, The loss during the training process.\nFigureR26 Training on binarized MNIST dataset using CNN. a, The processed MNIST dataset is trained on a CNN with the following architecture: a 14×14 input (downsampled and binarized MNIST dataset), followed by a two-stage feature extraction structure. The first stage employs a 16-channel 3×3 convolution (with batch normalization and ReLU activation) and 2×2 max pooling, while the second stage uses a 32-channel convolution. The final prediction is achieved through a flattening layer, a fully connected layer with 64 neurons, and a 10-dimensional classification layer, resulting in approximately 106,000 total parameters. b, The confusion matrix of the trained network, showing a recognition accuracy of 93.57%. c, The recognition accuracy during the training process. d, The loss during the training process.\nFigure R27. Training of the binarized Fashion_MNIST dataset on a fully connected network (FCN). a, The processed Fashion_MNIST dataset is trained on an FCN with the following architecture: a 14×14 input layer (representing the downsampled and binarized Fashion_MNIST dataset), a fully connected layer with 7 nodes, a ReLU nonlinear activation layer, a fully connected layer with 10 nodes, a Softmax nonlinear activation layer, and a 10-class classification output. b, The confusion matrix of the trained network, achieving an identification accuracy of 75.46%. c, The identification accuracy during the training process. d, The loss during the training process.\nFigure R28. Training on binarized Fashion_MNIST dataset using CNN. a, The processed Fashion- MNIST dataset is trained on a CNN with the following architecture: a 14×14 input (downsampled and binarized Fashion-MNIST dataset), followed by a two-stage feature extraction structure. The first stage employs a 16-channel 3×3 convolution (with batch normalization and ReLU activation) and 2×2 max pooling, while the second stage uses a 32-channel convolution. The final prediction is achieved through a flattening layer, a fully connected layer with 64 neurons, and a 10-dimensional classification layer, resulting in approximately 106,000 total parameters. b, The confusion matrix of the trained network, showing a recognition accuracy of 83.22%. c, The recognition accuracy during the training process. d, The loss during the training process.\nTable R11 Classification accuracy of FCN, CNN, and MN-CNN on binarized MNIST/Fashion-MNIST datasets\nNet MNIST Accuracy FASHION_MNSIT Accuracy\nMN-DNN\n92.83% 78.8%\n(this work)\nFCNN 85.97% 75.46%\nCNN 93.57% 83.22%\nPerformance comparison on other nonlinear DNNs\nA comparative analysis between our design and existing nonlinear DNN models demonstrates (as shown in Table R12) that the proposed network achieves object recognition accuracy on continuous grayscale datasets comparable to those reported in references [R28][R29], effectively validating the rationality of our network architecture. However, when the down sampling combined with binarization preprocessing method is applied to the more structurally complex Fashion-MNIST dataset, the recognition accuracy decreases by 4.9%, compared to a 1.97% reduction for the MNIST dataset (as discussed in response to Comment 2.4). This indicates that the preprocessing procedure leads to more substantial information loss in Fashion-MNIST. Notably, reference [R30] reports that nonlinear DNN model achieves approximately 80% accuracy in recognizing binarized Fashion- MNIST (28×28 input), a performance level consistent with our nonlinear network’s results, which further corroborates the effectiveness of our approach.\nTable R12 Accuracy Comparison with Nonlinear DNNs\nFashion Input\nNonlinear DNN MNSIT\nMNIST Dataset\nMN-DNN (this work) 92.81% 78.8% Binary\nMN-DNN (this work) 94.8% 83.7% Grayscale\nLarge-scale neuromorphic optoelectronic computing with a\n96% 84.6% Grayscale\nreconfigurable diffractive processing unit [R28]\nSingle-chip photonic deep neural network with forward-only training\n96% 87% Grayscale\n[R29]\nNonlinear optical encoding enabled by recurrent linear scattering [R30] - 80% Binary\nLow-power scalable multilayer optoelectronic neural networks enabled\n91.8% - Grayscale\nwith incoherent light [R30]\nHyper multiplexed integrated photonics–based optsical tensor processor\n91% - Grayscale\n[R32]\nIn summary, the above analysis demonstrates the effectiveness of our network, while the observed accuracy degradation on Fashion-MNIST can be attributed to information loss caused by down- sampling and binarization of the input data.\nReferences:\n[R28] Zhou T, et al. Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit. Nat Photonics 15, 367-373 (2021).\n[R29] Bandyopadhyay S, et al. Single-chip photonic deep neural network with forward-only training. Nat Photonics 18, 1335-1343 (2024).\n[R30] Xia F, et al. Nonlinear optical encoding enabled by recurrent linear scattering. Nat Photonics 18, 1067-1075 (2024).\n[R31]Song A, Murty Kottapalli SN, Goyal R, Schölkopf B, Fischer P. Low-power scalable multilayer optoelectronic neural networks enabled with incoherent light. Nat Commun 15, (2024).\n[R32] Ou S, et al. Hypermultiplexed integrated photonics–based optical tensor processor. Sci Adv 11, eadu0228.\nAction Taken:\n[Supplementary Note 11 Accuracy comparison with other networks]\nThe aforementioned results and analyses have been included in Supplementary Note 11.\nComment 2.6:\nFor dynamic gesture recognition, I do not agree with the “real-time” claim of the authors. For a system to be called real time, it needs to be much faster compared to the response time of humans. Typical systems that are quoted as “real-time” operate at over 30Hz refresh rates, which is much faster than the response of this system. Moreover, the response time of the presented system currently can only be inferred from Fig 6c, as no formal analysis is presented. Note that this is different from the 4.5ns delay for the metasurface layer shown in Fig 1. I suggest the authors actually quantify this response time, discuss factors contributing to it and comment on how the system can be made faster.\nResponse:\nThank you for your insightful question. The total system delay is 2.48 μs, comprising both the propagation delay in the MN-DNN and the latency introduced by the output detection circuitry. This corresponds to a refresh rate of 403 kHz, which far exceeds the conventional 30 Hz real-time benchmark and fully satisfies real-time processing requirements. In future implementations, integrating higher-speed ADC modules and FPGAs into the detection circuitry could further reduce the total latency to approximately 68.5 ns. The comprehensive analysis of system latency and the corresponding optimization approaches are provided in our response to Comment 1.2.\nThe voltage fluctuations observed in Fig. 6c arise from transient effects during posture transitions and are not indicative of the system’s intrinsic response delay.\nAction Taken:\n[Dynamic posture recognition by MN-DNN]\nThe total system latency is 2.48 μs, comprising both the propagation delay through the MN-DNN and the processing delay incurred by the output detection. This represents an at least three-order-of-magnitude reduction in latency compared to conventional camera systems coupled with digital neural network post-processing, which typically operate at millisecond-level latencies. By implementing a higher-speed output detection scheme (e.g., utilizing high-speed ADC modules and high-performance FPGAs), the total system latency could be further reduced to 66.5–68.5 ns (see Supplementary Note 27 for details).\nComment 2.7:\nWhile the threshold for activation is low, what is the actual power consumption of the active components (amplifiers, detectors, adders) per unit cell, and for the entire system? How does this compare to other DNN approaches when considering the entire network?\nResponse:\nThank you for your insightful question. Each nonlinear unit in our system demonstrates a total power dissipation of approximately 429 mW, with the RF amplifier contributing 150 mW, the RF detector 53 mW, and the voltage adder 226 mW. The overall system power consumption is 252.352 W. The proposed system achieves an energy efficiency of 0.25 Tops/W, which is competitive with existing approaches. Further power reduction can be attained by integrating ultra-low-power components such as the MAX2659 RF amplifier (power consumption: 6.56 mW), LT5537 RF detector (40.5 mW), and LPV821 op-amp (2.844 μW), potentially reducing total power consumption by~90%. Notably, our architecture exhibits strong scaling advantages. When scaled to 56×56 nonlinear units per layer (with linear layers expanding to 88×88), energy efficiency improves to 4 Tops/W. This scalability makes our approach particularly compelling for large-scale neural network deployment where power efficiency is critical. For detailed calculations and comparative analysis with other DNN approaches, please refer to our response to Comment 1.6.\nComment 2.8:\nThe manuscript does not provide a detailed discussion of the number of trainable parameters, which is a crucial aspect for understanding the model's complexity and scalability. It would be beneficial for the authors to explicitly mention the total number of trainable parameters and offer a comparison with those typically found in conventional neural networks. Furthermore, the number of trainable parameters associated with both the linear and non-linear layers should be clearly specified to provide a more comprehensive understanding of the model's architecture and its computational demands. A more thorough discussion in this regard would improve the transparency of the model's design and facilitate a clearer comparison with existing approaches in the literature.\nResponse:\nWe sincerely thank the reviewer’s valuable feedback. In response to the comments regarding the trainable parameters in our model, we have conducted a comprehensive review and analysis. Below, we provide a detailed clarification:\nModel Architecture and Parameter Efficiency\nThe linear component of our model comprises three linear layers, each containing 22×22 = 484 trainable phase parameters constrained within the range [0, 2π]. This results in a total of 1,452 trainable parameters for the linear network, as shown in Fig.R29. In contrast, the nonlinear network extends this structure by introducing three nonlinear layers. Each nonlinear layer consists of 14×14 computational units that operate under a shared state, meaning all units within a layer utilize the same nonlinear threshold and slope. Consequently, each nonlinear layer requires training only two key parameters, the threshold and slope of the ReLU activation function. This design enables the nonlinear network to achieve complex transformations while introducing only six additional parameters, yielding a total of 1,458 trainable parameters, as shown in Fig.R30.\nFigure R29. Architecture and trainable parameters of the linear network. a, Schematic diagram of the linear network architecture. b, Equivalent network representation with corresponding trainable parameters per layer. c, Tabulated summary of network layers and their associated trainable parameters.\nFigure R30. Architecture and trainable parameters of the nonlinear network. a, Schematic diagram of the nonlinear network architecture. b, Equivalent network representation with corresponding trainable parameters per layer. c, Tabulated summary of network layers and their associated trainable parameters.\nComparative Efficiency with Traditional Neural Networks\nOur model demonstrates remarkable parameter efficiency compared to conventional deep learning architectures. For instance, standard fully connected networks for MNIST classification typically require tens to hundreds of thousands of parameters, while even lightweight CNNs such as LeNet-5 demand approximately 60,000 parameters. In contrast, our architecture achieves competitive performance with only ~1,500 parameters—an order of magnitude reduction in model size. This extreme parameter efficiency positions our model within the TinyML (Tiny Machine Learning), domain, yet it retains high recognition accuracy while significantly improving computational efficiency.\nScalability and Flexibility\nNotably, our architecture exhibits exceptional scalability. By adjusting the network dimensions (e.g., expanding each linear layer to 56×56 units), the total trainable parameters can be increased by ~9,414, reaching a scale (~10,000 parameters) comparable to conventional fully connected networks. This flexibility allows the model to maintain its lightweight advantages while adapting to more demanding computational tasks without substantial increases in inference time. Such scalability ensures broad applicability across varying complexity requirements while preserving efficiency.\nAction Taken:\n[Handwritten digits classification by MN-DNN]\nThe nonlinear network consists of 1,458 trainable parameters, with 484 parameters per linear layer and 2 parameters (threshold and slope) per nonlinear layer, yielding a total of 484 × 3 + 2 × 3 = 1,458. In comparison, standard fully connected networks for MNIST classification typically require tens to hundreds of thousands of parameters, whereas our model achieves an order-of-magnitude reduction with only ~1,500 parameters. Future work could explore scaling the network dimensions (e.g., expanding linear layers to 56 × 56 units), which would increase the parameter count by ~9,414, bringing it closer to conventional fully connected networks (~10,000 parameters) while maintaining efficiency.\nComment 2.9:\nThe training and test process of the MN-DNN model on both the MNIST and Fashion-MNIST datasets lacks clarity regarding the sample sizes used. On page 11, line 269 of the main text, it is mentioned that 200 samples were randomly selected for the MNIST dataset. However, considering that MNIST consists of 70,000 samples, the selection of only 200 samples would be insufficient for a robust evaluation. A larger test sample size would provide more reliable and representative accuracy metrics. Was there a specific reason why such a small sample set was used? In this case, I would recommend that the authors increase the test sample size and re-evaluate the accuracy values accordingly. Additionally, the test sample size for the Fashion-MNIST dataset is not specified, which should be addressed to ensure the comparability and reliability of the results. Clear documentation of the sample sizes for both datasets is essential to assess the validity of the reported findings.\nResponse:\nWe thank the reviewer for this constructive suggestion. Following your advice, we have extended our evaluation by testing on 5,000 samples from both MNIST and Fashion-MNIST datasets to ensure robustness. The additional results and discussions have been added to the manuscript and supplementary materials accordingly.\nAction Taken:\n[Handwritten digits classification by MN-DNN]\nTo comprehensively evaluate network performance, we systematically assessed the recognition accuracy of both networks using 5,000 randomly selected images. In the experimental setup, a rapid detection device (see Supplementary Note 13) replaced the conventional two-dimensional near-field scanning stage, enabling efficient energy measurements across ten specific regions of the output field for each image. This approach significantly enhanced detection throughput, allowing rapid acquisition of classification results for large-scale image datasets. The MNIST rapid classification experiments conducted with this device are shown in Supplementary Video 1. Experimental results demonstrated that the nonlinear network achieved a classification accuracy of 92.4% on the 5,000 MNIST images (compared to a simulated accuracy of 92.81%), exhibiting a clear advantage over the linear network (88.5% accuracy, Fig. 3f).\nFig. 3 | Handwritten digits classification. a Illustration of the MNIST dataset for recognition, including 10 classes of handwritten digits (0-9). b Schematic of the input image encoding process, where binarized images are translated into transmission coefficients of the input layer metasurface. c Photo of the testing scenario, where a horn antenna illuminates the input layer, directing the EM waves carrying the image information into MN-DNN. The output field distribution is scanned using a two-dimensional near-field scanner. d Comparison of linear and nonlinear networks is facilitated by two optimized designs: a nonlinear network with interleaved 3 linear and 3 nonlinear layers, and a linear network with only 3 linear layers. e t-SNE visualization of test set outputs for both networks, demonstrating that the nonlinear network achieves more distinct class separation and tighter intra-class clustering. f Confusion matrix for 5000 test images, showing the 92.4% and 88.5% accuracy of the nonlinear network and linear network. g Example output field distributions generated by the linear and nonlinear networks for one simple image, demonstrating that both networks successfully focus on the target region. h Example comparative output field distributions for two complex images from both networks, with the nonlinear network focusing on correct regions and the linear network on incorrect regions.\n[Fashion-MNIST classification by MN-DNN]\nDuring experimental evaluation, a set of 5,000 randomly sampled Fashion-MNIST test images were intensity-encoded and processed through both nonlinear and linear network configurations. Recorded recognition accuracies reached 77.0% and 72.5% for the nonlinear and linear networks, respectively.\n[Supplementary Note 13: Fast Detection Device for Output Field Distribution]\nSupplementary Figure S22 | Output plane power detection device. a, Photograph of the MN-DNN system’s output plane detection device. b, Back view of the output plane detection device. c, Photograph of an individual RF power detection circuit board. d, Photograph of an individual detection circuit board connected to the MCU and LED.\nFor fast output detection, we replace the conventional two-dimensional near-field scanning platform with a rapid output plane detection device, which measures ten regions on the output plane, as shown in Supplementary Figure 22a. The output plane is positioned 10 cm away from the last layer of the hidden layer. A large acrylic plate is supported at the output plane, with 10 designated areas each equipped with a patch antenna and detector to detect the power distribution across the output plane. Additionally, an MCU and an LED (not shown in the figure, located on the side) are mounted on the setup to provide real-time comparison of the energy in the 10 areas and to display the classification results.\nSupplementary Figure S22b depicts a back view of the output plane detection apparatus. 10 patch antennas are secured to an acrylic board using plastic screws, positioned at 10 distinct regions designated for output energy detection. The central frequency of these patch antennas matches the operational frequency of the MN-DNN, which is 5.8GHz. Each patch antenna is interfaced with a circuit board designed to perform RF amplification and RF detection, which transforms the RF energy captured by the patch antenna into an equivalent direct current (DC) voltage. The detection circuits are mounted on L-shaped acrylic panels using plastic screws, which are then affixed to the larger acrylic panel via the same fastening technique. The electrical connections for each detection circuit are configured in parallel to maintain a uniform DC voltage bias across all circuits, thus standardizing the performance characteristics of each detection circuit. Supplementary Figure S22c presents a photographic representation of a detection circuit that incorporates a two-stage amplifier and a detector component.\nSupplementary Figure S22d provides a photographic illustration of the interface between the detection circuit, the microcontroller unit (MCU) Arduino DUE R3, and the light-emitting diode (LED). The output voltage from the detection circuit is connected to the MCU's analog-to-digital converter (ADC) input for digitization. This digital signal is then subjected to a comparison process across 10 discrete ports. Based on the port exhibiting the highest voltage reading, the MCU generates a corresponding code to regulate the color of the LED. Additionally, the classification result associated with the port exhibiting the highest voltage value is returned to the computer in real-time through serial communication with the MCU Arduino DUE R3. \nSupplementary Figure S23 illustrates the complete equivalent circuit of the high-speed detection system. The device operates as follows: A host computer communicates with an STM32 MCU via a serial interface, which orchestrates the encoding of the input plane. By precisely modulating the voltage applied to each unit cell in the input layer, the input image is spatially encoded onto the transmission coefficients of the metasurface. EM waves then propagate through the encoded input plane, traversing the diffractive neural network before reaching the output plane. At the output plane, RF signals detected by an array of 10 patch antennas undergo amplification via a two-stage RF amplifier. Each amplification stage provides tunable gain (0–10 dB) through voltage-controlled (V ) regulation, yielding a total adjustable gain of 0–20 dB. This wide dynamic range ensures CTRL robust detection of RF signals across varying intensities. The amplified signals are synchronously sampled and digitized through the MCU’s ADC channels. Classification is determined by identifying the channel with the highest output voltage, with results visually indicated via an LED display and simultaneously logged on the host computer for further analysis. Using this optimized detection system, we demonstrate efficient classification of 5,000 test images using both linear and nonlinear diffractive network configurations, with the nonlinear network achieving superior performance (92.4% accuracy) compared to its linear counterpart (88.5%).\nSupplementary Figure S23| Schematic of the equivalent circuit for the rapid detection system.\nComment 2.10:\nSome information should be clarified regarding the robustness of posture recognition. The authors collected data from 4 volunteers under controlled settings (angles, distances, etc.). How robust is the system to variations in various changes in data such as clothing, body size/shape, different individuals not in the training set, or other complex background environments? The use of only 8 spatial sampling points seems sparse; is it not possible to use more sampling points? Also, how sensitive is the system performance to mispositioning of the person relative to these points?\nResponse:\nWe appreciate the reviewer's insightful comments regarding system robustness. We will address each question in sequence.\nQuestion 1: How robust is the system to variations in data, such as clothing, body size/shape, different individuals not included in the training set, or other complex background environments?\nTo address the question, we conduct comprehensive experiments to evaluate the model's generalization capacity across diverse anthropometric variations and environmental conditions. For subject variability assessment, we include four new participants who did not participate in the training data collection with distinct physical characteristics (height range: 160-190 cm; weight range: 50-80 kg) wearing various clothing types (nylon pants, cotton T-shirts, and denim shorts) that differ from the original training dataset (detailed information of the four new participants is provided in Table R13). Environmental robustness evaluation involves two experimental conditions: one using anechoic chamber materials to establish baseline performance, and another introducing real-world complexity by removing shielding materials and allowing interference from ambient 5.8 GHz WIFI signals and multipath reflections (Fig. R31).\nTable R13. Distribution of participant characteristics in external validation testing\nWeight\nData Subjects Clothing Height(m) Gender Environment\n(Kg)\nExternal evaluator Cotton T-shirt and denim Controlled and\n1.7~1.8 60~70 Female\n1 short cluttered\nExternal evaluator Cotton T-shirt and cotton Controlled and\n1.8~1.9 70~80 Male\n2 pant cluttered\nExternal evaluator Cotton T-shirt and poly Controlled and\n1.6~1.7 50~60 Male\n3 pant cluttered\nExternal evaluator Controlled and\nPoly T-shirt and poly pant 1.7~1.8 70~80 Male\n4 cluttered\nFigure R31．Experimental setup for robustness evaluation. New external participants perform gesture recognition tests under both (a) controlled (anechoic) and (b) complex (cluttered with WiFi interference) environmental conditions.\nFigure R32. Photographic documentation of posture validation. External participants sequentially perform all eight test postures.\nFigure. R33. Confusion matrix for external validators.\nDuring the experiment, participants used the MN-DNN system to sequentially complete recognition tasks for eight predefined hand gestures (Fig.R32). The system achieved classification by directly processing human-scattered waves, with the classification results displayed by the color of LED lights on the output plane. Through multiple rounds of testing and statistical analysis, the system achieved an average recognition accuracy of 87% across new participants and varying environmental conditions (Fig.R33). Although this represents a decrease compared to the 93.4% accuracy observed under ideal conditions, the results still demonstrate the system's adaptability and robustness to factors such as variations in clothing, body shape differences, and interference from complex environments.\nFuture studies should focus on enhancing data collection in two key dimensions: (1) expanding the participant cohort to incorporate greater anthropometric diversity (including varying heights and body builds), and (2) diversifying the data acquisition environments to encompass a broader spectrum of real-world conditions. The development of such a more comprehensive and robust dataset will significantly improve the system's adaptability to individual differences and environmental variability.\nQuestion 2: The use of only 8 spatial sampling points seems sparse; is it not possible to use more sampling points?\nThank you for your insightful comment regarding the number of data collection points. Indeed, increasing the number of acquisition points could offer additional benefits. Our initial choice of 8 collection points was carefully considered to achieve an optimal balance between data acquisition efficiency and recognition accuracy. While the primary objective of this study is to demonstrate the feasibility of MN-DNN for posture recognition, we fully acknowledge that incorporating more collection points could potentially enhance both accuracy and robustness, particularly in complex real-world scenarios.\nWe highly value this constructive suggestion and have conducted new experiments involving additional collection points for data collection and analysis. We increase the number of sampling points from 8 to 21 in an optimized 3-5-5-5-3 distribution (Fig.R34a, b). The revised system incorporates horn antennas, a multi-channel RF matrix switch, and a vector network analyzer, enabling high-efficiency multi-channel signal acquisition via coaxial cables (Fig.R34 c, d).\nFigure R34. The data acquisition system after adding sampling points. a, Spatial distribution of the 21 sampling points. b, Physical image of the 21 spatial sampling points. c, Photo of the data acquisition system, including the horn and receiving antenna array. d, Photo of the data acquisition system, including the PC, RF matrix switch, and VNA.\nTo ensure robust generalization across diverse real-world conditions, we recruited eight participants (6 male, 2 female) exhibiting significant variations in body type (height: 1.5~1.9 m, weight: 40~90 kg) and clothing materials (including cotton T-shirts, yarn long sleeves, and polyester pants), see Table R14 for details. Each participant performed multiple repetitions of the full posture set under controlled and cluttered environments allowing interference from ambient 5.8 GHz WIFI signals and multipath reflections (Fig. R35 and Fig.R36).\nFigure R35. Data acquisition photographs. Eight trainers performing data collection for eight different actions.\nTable R14. Distribution of participant characteristics in new data collecting\nData Weight\nClothing Height(m) Gender Environment\nSubjects (Kg)\nControlled and\nTrainer 1 Cotton T-shirt and cotton pant 1.8~1.9 70~80 Male\ncluttered\nYarn long sleeve and denim Controlled and\nTrainer 2 1.5~1.6 40~50 Female\nshort cluttered\nControlled and\nTrainer 3 Cotton T-shirt and poly pant 1.7~1.8 60~70 Male\ncluttered\nControlled and\nTrainer 4 Nylon T-shirt and denim pant 1.8~1.9 70~80 Male\ncluttered\nControlled and\nTrainer 5 Nylon T-shirt and poly short 1.8~1.9 80~90 Male\ncluttered\nControlled and\nTrainer 6 Poly T-shirt and poly pant 1.7~1.8 70~80 Male\ncluttered\nControlled and\nTrainer 7 Cotton T-shirt and nylon pant 1.7~1.8 60~70 Male\ncluttered\nControlled and\nTrainer 8 Wool T-shirt and cotton pant 1.6~1.7 50~60 Female\ncluttered\nFigure R36. Photos of data acquisition. a, Data collection in the complex environment. b, Data collection at different positions.\nWhen trained on the newly constructed dataset (using an 80:20 training-validation split), the nonlinear DNN achieved a recognition accuracy of 92.7%. This optimized data acquisition protocol delivers comprehensive performance improvements: (1) more spatial sampling density captures richer features; (2) increased participant diversity ensures broader coverage of human physiological characteristics; (3) positional variation during data collection strengthens system robustness.\nParticularly noteworthy is the performance for test subject (external validator 2/trainer 1) with body types substantially different from the initial four volunteers, the recognition accuracy shows a dramatic improvement from 78% to 91.5%, conclusively demonstrating the enhanced robustness of our modified dataset against individual variability.\nDue to current hardware constraints, in the manuscript, we implement MN-DNN for simple experimental validation. Future work will involve fabricating new passive phase-modulation metasurfaces with reprogrammable nonlinear layer voltages to enable hardware implementation of DNN architectures trained on this enhanced dataset.\nQuestion 3: Also, how sensitive is the system performance to mispositioning of the person relative to these points?\nFor localization error assessment, we performed multi- participants, multi-round acquisitions of eight actions and statistically evaluated positioning accuracy at various locations. As illustrated in Fig.R37, we systematically evaluate the recognition accuracy across different positions for all test subjects, with the quantitative results summarized in Table R15. The recognition accuracy is influenced by the position of the human body, exhibiting a more significant decline as the movement distance increases. Notably, the reduction in accuracy is less substantial for forward and backward movements, as the dataset collection process incorporated these directional variations.\nFigure R37. Experimental images showing participants at different locations.\nTable R15 Test accuracy at different locations (8 collection points)\nLocation x=0, y=0 x=-10cm x=-20cm x=-30cm x=10cm x=20cm\nAccuracy 93% 85.2% 61.6% 27.2% 87.6% 63.4%\nLocation x=30cm y=-10cm y=-20cm y=10cm y=20cm\nAccuracy 32.1% 91.5% 87.2% 90.5% 85.6%\nTable R16 Simulation accuracy at different locations (21 collection points)\nLocation x=0, y=0 x=-10cm x=-20cm x=-30cm x=10cm x=20cm\nAccuracy 94.2% 92.5% 87.6% 70.8% 93.1% 85.0%\nLocation x=30cm y=-10cm y=-20cm y=10cm y=20cm\nAccuracy 71.2% 93.6% 90.3% 92.6% 89.7%\nFurthermore, we evaluate the simulated accuracy across different positions using the newly collected dataset with 21 sampling points (location information is recorded). The quantitative results are summarized in Table R16. The results demonstrate that, compared to the accuracy achieved with only 8 sampling points, the 21-point sampling method achieves higher recognition accuracy under positional offsets, indicating improved location robustness. This improvement primarily stems from both the increased number of sampling points and the incorporation of more diverse samples during data collection (including data from different individuals at various locations).\nAction Taken:\n[Posture recognition]\nTo evaluate the system's robustness, we conduct tests with four additional participants (exhibiting diverse body shapes and clothing) in complex backgrounds, none of whom had participated in the initial data collection. The results showed an average recognition accuracy of 87%. Future improvements could focus on expanding the data collection scope by incorporating more sampling points and recruiting additional participants with varied body shapes, clothing styles, locations, and environmental conditions. This expansion would enhance the system's adaptability and robustness across diverse scenarios (see Supplementary Note 26 for detailed analysis).\n[Methods: Data collection for posture recognition.]\n…To improve dataset robustness, we increase sampling points from 8 to 21. Eight participants with diverse body types and clothing materials perform eight postures repeatedly under both controlled and interference-rich conditions (5.8 GHz WIFI with multipath reflections). As shown in Supplementary Note 26, this enhanced dataset demonstrates significantly improved robustness in simulations.\n[Supplementary Note 26: Robustness and enhancement of posture recognition]\nThe aforementioned results and analyses have been included in Supplementary Note 26.\nREVIEWER #3\nWe sincerely appreciate the reviewer for the time and effort invested in reviewing our manuscript and offering a wealth of insightful comments and thoughtful suggestions. These comments and suggestions are extremely valuable to enhance the quality of our manuscript. Following your constructive guidance, we carefully revised the paper to improve its clarity and comprehensibility.\nREVIEWER #4\nWe sincerely appreciate the reviewer for the time and effort invested in reviewing our manuscript and offering a wealth of insightful comments and thoughtful suggestions. These comments and suggestions are extremely valuable to enhance the quality of our manuscript. Following your constructive guidance, we carefully revised the paper to improve its clarity and comprehensibility. Below, we provide our one-by-one responses (in black fonts) to the referees’ comments (in blue fonts).\nRemarks to the Author:\nThe paper by Ning et al presents a highly novel contribution to the field of diffractive neural networks (DNNs) by introducing a multilayer nonlinear diffractive neural network (MN-DNN) operating in the microwave band. This system leverages programmable metasurfaces integrated with RF amplifiers, detectors, and voltage adders to implement a tunable ReLU-like nonlinear activation function with an impressive 4.5 ns latency, significantly faster than the millisecond/microsecond latencies of conventional optoelectronic methods. The MN-DNN's ability to adjust activation thresholds and slopes via bias voltages enables real-time reconfiguration, addressing the rigidity of prior optical DNNs. Additionally, its low-power nonlinear processing and direct electromagnetic (EM) wave processing for tasks like human posture classification set it apart from existing approaches, which often rely on high-energy inputs or digital encoding. The research represents an impactful contribution to both research and practical deployment. I am happy to recommend its publication in Nature Communications provided the authors could address the following points.\nResponse:\nThank you very much for your kind support and encouraging comments. We have carefully addressed your questions point by point. Please refer to the details as below.\nComment 4.1:\nIn the introduction, the authors described the advantages of their DNNs in comparison to other approaches with nonlinear activation, including the short latency and low threshold. The authors also made comparison of the accuracy to the linear networks (something like 93% vs 89%). The authors should provide comparison of the accuracy levels with other nonlinear DNNs.\nResponse and Action Taken:\nWe sincerely appreciate the reviewer’s valuable comment and suggestion. In response to the raised concerns, we have supplemented the Introduction section with a systematic comparative analysis between the proposed deep neural network model and other mainstream nonlinear deep neural networks in terms of classification accuracy. The results demonstrate that our method achieves comparable accuracy (93% on MNIST) to current state-of-the-art nonlinear DNN models (with accuracy ranging approximately from 91% to 96%). To further enhance this comparative analysis, we have provided a detailed tabulation of the evaluation metrics in the Table R17. Additionally, in the Discussion section, we have explored potential strategies for improving model performance, including optimizing the feature input mechanism at the network input layer and refining the hierarchical design of the network architecture.\n[Introduction]\nThus, MN-DNN substantially improves information extraction, outperforming linear networks by over 4.2% in classification accuracy on both MNIST and Fashion-MNIST datasets (Figs. 1d–1e) and matching the performance of state-of-the-art nonlinear DNNs.\n[Supplementary Table]\nTable R17. Accuracy Comparison with Nonlinear DNNs\nFashion Input\nNonlinear DNN MNSIT\nMNIST Dataset\nMN-DNN (this work) 93% 78.8% Binary\nMN-DNN (this work) 94.8% 83.7% Grayscale\nLarge-scale neuromorphic optoelectronic computing with a\n96% 84.6% Grayscale\nreconfigurable diffractive processing unit [R33]\nSingle-chip photonic deep neural network with forward-only\n96% 87% Grayscale\ntraining [R34]\nNonlinear optical encoding enabled by recurrent linear\n- 80% Binary\nscattering [R35]\nLow-power scalable multilayer optoelectronic neural\n91.8% - Grayscale\nnetworks enabled with incoherent light [R36]\nHyper multiplexed integrated photonics–based optsical tensor\n91% - Grayscale\nprocessor [R37]\nReferences:\n[R33] Zhou T, et al. Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit. Nat Photonics 15, 367-373 (2021).\n[R34] Bandyopadhyay S, et al. Single-chip photonic deep neural network with forward-only training. Nat Photonics 18, 1335-1343 (2024).\n[R35] Xia F, et al. Nonlinear optical encoding enabled by recurrent linear scattering. Nat Photonics 18, 1067-1075 (2024).\n[R36]Song A, Murty Kottapalli SN, Goyal R, Schölkopf B, Fischer P. Low-power scalable multilayer optoelectronic neural networks enabled with incoherent light. Nat Commun 15, (2024).\n[R37] Ou S, et al. Hypermultiplexed integrated photonics–based optical tensor processor. Sci Adv 11, eadu0228.\n[Discussion]\nFuture architectural improvements could target two key aspects to enhance recognition accuracy: Firstly, integration of a digital-to-analog converter (DAC) with the input-layer metasurface to enable continuous grayscale inputs, thereby improving information fidelity and subsequent feature extraction accuracy (Supplementary Note 30). Additionally, systematic scaling of network dimensions through increased metasurface unit density and additional layers to strengthen nonlinear processing capacity, ultimately boosting pattern discrimination performance.\nComment 4.2:\nThere seems to be no mention of the operating frequency of the metasurface DNN. Would it be possible to increase the operating frequency so that the system can be more compact?\nResponse:\nWe sincerely appreciate the reviewer’s valuable comments. First, we would like to apologize for the insufficient clarity regarding the operating frequency in the original manuscript. In fact, the working frequency of the MN-DNN is 5.8 GHz. Following the reviewer's suggestion, we have supplemented detailed explanations of the relevant frequency parameters in the revised manuscript.\nRegarding to the reviewer’s suggestion to enhance system compactness by increasing the operating frequency, we fully agree with this perspective. From a technical implementation standpoint, elevating the system's operating frequency can indeed effectively reduce device dimensions, thereby achieving overall system miniaturization. Specifically, this optimization can be advanced synergistically from two dimensions: active devices and passive structures. For active devices, adopting higher-frequency RF components (such as millimeter-wave/terahertz low-noise amplifiers based on InP HEMT [R38], SiGe [R39-R40], or CMOS [R38] processes, as well as high-frequency detectors [R42-R43] like the PE80T3002 or VDI ZBD) can readily extend the system's operating frequency to the millimeter-wave or even terahertz range. For passive structures, through reasonable scaling design, the existing antenna and metasurface unit structures can be fully adapted to higher operating frequency bands. This holistic optimization approach not only maintains system performance but also significantly enhances integration density.\nBased on the reviewers' suggestions, we will prioritize increasing the operating frequency as a key direction for future research, focusing on exploring the feasibility of achieving high-performance compact nonlinear DNN systems in the millimeter-wave and even terahertz frequency bands. We once again express our gratitude for the reviewers' constructive feedback.\nReferences:\n[R38] Hu L, et al. A 110-170 GHz Wideband LNA Design Using the InP Technology for Terahertz Communication Applications. Micromachines 14, (2023).\n[R39] Singh SP, Rahkonen T, Leinonen ME, Parssinen A. A 290 GHz Low Noise Amplifier Operating above $f_{max}/2$ in 130 nm SiGe Technology for Sub-THz/THz Receivers. In: 2021 IEEE Radio Frequency Integrated Circuits Symposium (RFIC)) (2021).\n[R40] Singh SP, Rahkonen T, Leinonen ME, Parssinen A. Design Aspects of Single-Ended and Differential SiGe Low-Noise Amplifiers Operating Above fmax/2in Sub-THz/THz Frequencies. IEEE J Solid-State Circuits 58, 2478-2488 (2023).\n[R41] Wang L, Shen Y, Qian Y, Ding Y, Hu S. A 220-GHz LNA With 9.7-dB Noise Figure and 24.6- dB Gain in 40-nm Bulk CMOS. IEEE Trans Circuits Syst II Express Briefs 72, 113-117 (2025).\n[R42] Ji DF, Niu B, Tao HQ, Chen TS, Wang WB. A D Band Zero Bias Detector Chip Using Schottky Diode. In: 2022 Photonics & Electromagnetics Research Symposium (PIERS)) (2022).\n[R43] Ludwig F, et al. Graphene field-effect transistors as THz detectors: Distinguishing between resistive self-mixing and the hot-carrier thermoelectric effect. In: 2023 48th International Conference on Infrared, Millimeter, and Terahertz Waves (IRMMW-THz)) (2023).\nAction Taken：\n[Abstract]\n…In this work, we propose a programmable multilayer full-space nonlinear neural network operating at microwave frequencies (5.8 GHz). …\n[Introduction]\n…Here, we propose a multilayer nonlinear diffractive neural network (MN-DNN) with the programmable and fast rectified linear unit (ReLU) activation function operating at 5.8 GHz. …\n[Architecture of nonlinear metasurface unit]\n…Optimized for 5.8 GHz operation, the patch antenna exhibits a side length dimension of D = 7.57 mm to achieve proper resonance characteristics. …\n[Handwritten digits classification by MN-DNN]\n…The optimization procedure explicitly accounts for the 5.8 GHz horn antenna's field distribution at the input plane under test conditions (Supplementary Note 10), to reduce discrepancies between simulations and measurements.…\n…Both the linear and nonlinear prototypes are excited by a 5.8 GHz horn antenna. …\n[Static posture recognition by MN-DNN]\n…The system employs a standard horn antenna operating at 5.8 GHz to illuminate the test person. Posture information is extracted using an eight-element patch antenna array (5.8 GHz resonant frequency, 3-2-3 arrangement) that samples the scattered field distributions from the human body. …\n[Discussion]\n…The system can be extended to millimeter-wave and terahertz frequencies by integrating high- frequency semiconductor devices (e.g., InP/SiGe amplifiers10-13 or Schottky diode14-15 detectors) with scalable metasurface designs. This configuration maintains integration density while supporting higher-frequency operation, resulting in a more compact and highly integrated system suitable for miniaturization…..\nComment 4.3:\nThe authors should discuss the key factors that set the lower limit of latency, and how it can be further reduced.\nResponse:\nWe thank the reviewer for their insightful discussion and constructive feedback on this key technical challenge.\nLower limit of latency\nThe fundamental lower bound of system latency is governed by two principal factors: the hardware response characteristics of the nonlinear layer and the EM wave propagation delay in air.\n(1) Hardware-induced latency arises from the cumulative delays in the RF amplifier, RF detector, and voltage adder in the nonlinear layer. These are intrinsically constrained by semiconductor physics and parasitic effects in circuit packaging, representing a fundamental physical limitation. However, such delays can be partially reduced through advancements in device fabrication and soldering techniques.\n(2) Propagation delay is dictated by the time of flight of EM waves between the input and output layers, with its value depending on the distance between the input and output layers. The delay can be mitigated through design optimizations, such as higher operational frequencies or refined interlayer architectures.\nThe interplay of these two factors determines the theoretical minimum latency achievable in the system.\nStrategies for further latency reduction\nTo address the challenge of further reducing system latency, we propose a comprehensive optimization framework encompassing device-level enhancements, nonlinear unit structural refinements, and system-level architectural improvements.\n(1) Device-Level Optimization\nThe adoption of high-performance components is critical for minimizing latency at the nonlinear unit level. By integrating ultra-fast operational amplifiers with slew rates exceeding 500 V/μs (response time <200 ps) and planar-deposited n-type silicon Schottky barrier diode detectors (response time <220 ps), alongside replacing conventional RF amplifiers with low-latency RF diode switches, we would achieve a single nonlinear unit delay below 426 ps (Table R18). This approaches the theoretical lower bound for response times in current nonlinear layer implementations.\nTable R18. Estimated response time of the nonlinear unit using ultrafast devices\nDetector Operational\nElements Schottky diode Overall\nresponse amplifier\nTime ~220ps ~200ps ~6 ps ~426 ps\nReference [R44] [R45] [R46] -\n(2) Nonlinear Unit Structural Optimization\nRelaxing the reconfigurability constraints of activation functions enables significant simplification of nonlinear layer. By eliminating operational amplifiers and retaining only essential nonlinear components, such as detectors and Schottky diodes (Fig. R38), we could further reduce single-layer latency to ~226 ps. This streamlined architecture maintains functionality while substantially improving temporal performance.\nFigure R38. Simplified Architecture of Nonlinear Unit.\n(3) System-Level Optimization\nMoreover, in the design of the system architecture, propagation delay can be decreased by reducing the distance between the input and output layers of the nonlinear diffractive neural network. This reduction is attainable through strategies such as elevating the operational frequency and refining the interlayer configuration, including decreasing layer spacing and streamlining the number of layers, which collectively shorten the signal transmission path.\nOverall, the proposed multi-scale optimization strategy, including spanning device selection, nonlinear unit structural refinement, and architectural redesign, provides a systematic approach to achieving sub-nanosecond system latency.\nReferences:\n[R44] Wang L, Fei X, Tang XH, Wu SY. A fast-speed pulse detector based on Si-Schottky diode. In: 2008 Asia-Pacific Microwave Conference) (2008).\n[R45] Haque MM, Chowdhury MIB. Effects of the field-dependence of the carrier mobility on the base transit time of SiGe HBTs having non-uniformly doped base. In: 2015 International Conference on Advances in Electrical Engineering (ICAEE)) (2015).\n[R46] Yang SJ, et al. Ultrafast 27 GHz cutoff frequency in vertical WSe2 Schottky diodes with extremely low contact resistance. Nat Commun 11, (2020).\nAction Taken:\n[Discussion]\nThe system latency can be further reduced through multiple optimization approaches (see Supplementary Note 28 for details). By employing high-performance active components in the nonlinear layer, the time delay can be optimized from the nanosecond down to the picosecond range. For example, using operational amplifiers with high slew rates reduces computational delay to below 200 ps, while fast-response diodes (e.g., rapid Schottky barrier diode detectors) as RF detectors shorten the delay to within 220 ps. Further improvements can be achieved by replacing conventional RF amplifiers with high-speed RF Schottky diode switches (~6 ps), enabling superior response speeds. Theoretical calculations indicate that the single-layer delay can be confined to within 426 ps, comparable to the fastest reported nonlinear layers. With a moderate trade-off in reconfigurability, this delay can be further reduced to 226 ps. Additionally, propagation delay can be effectively minimized by optimizing the nonlinear network architecture, for instance, through increased operational frequency and reduced interlayer spacing, thereby compressing the radial distance between input and output planes.\n[Supplementary Note 28: Methods for further reducing latency]\nThe aforementioned results and analyses have been included in Supplementary Note 28.\nReferences:\n[R1] Shahroudy A, Liu J, Ng T-T, Wang G. NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)) (2016).\n[R2] Cheng X, Zhang L, Tang Y, Liu Y, Wu H, He J. Real-Time Human Activity Recognition Using Conditionally Parametrized Convolutions on Mobile and Wearable Devices. IEEE Sens J 22, 5889-5901 (2022).\n[R3] Wang B, Huang L, Hoai M. Active Vision for Early Recognition of Human Actions. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)) (2020).\n[R4] Leightley D, Yap MH, Coulson J, Barnouin Y, McPhee JS. Benchmarking human motion analysis using kinect one: An open source dataset. In: 2015 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA)) (2015).\n[R5] Moon G, Chang JY, Lee KM. Camera Distance-Aware Top-Down Approach for 3D Multi- Person Pose Estimation From a Single RGB Image. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV)) (2019).\n[R6] Zhen J, et al. SMAP: Single-Shot Multi-person Absolute 3D Pose Estimation. In: Computer Vision – ECCV 2020 (eds Vedaldi A, Bischof H, Brox T, Frahm J-M). Springer International Publishing (2020).\n[R7] Jin L, et al. Single-Stage is Enough: Multi-Person Absolute 3D Pose Estimation. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)) (2022).\n[R8] Wang Z, Nie X, Qu X, Chen Y, Liu S. Distribution-Aware Single-Stage Models for Multi- Person 3D Pose Estimation. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)) (2022).\n[R9] Piergiovanni AJ, Angelova A, Ryoo MS. Tiny Video Networks. Applied AI Letters 3, (2022).\n[R10] Osokin D. Real-time 2D Multi-Person Pose Estimation on CPU: Lightweight OpenPose. In: Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods) (2019).\n[R11] Bazarevsky V, Grishchenko I, Raveendran K, Zhu TL, Zhang F, Grundmann MJA. BlazePose: On-device Real-time Body Pose tracking. abs/2006.10204, (2020).\n[R12] Imtiaz Hossain M, Akhter S, Delowar Hossain M, Seon Hong C, Huh E-N. Multi-Person 3D Pose Estimation in Mobile Edge Computing Devices for Real-Time Applications. In: 2023 International Conference on Information Networking (ICOIN)) (2023).\n[R13] A Distributed Real-time 3D Pose Estimation Framework based on Asynchronous Multiviews. KSII Trans Internet Inf Syst 17, (2023).\n[R14] THS3201: 1.8-GHz, LOW DISTORTION, CURRENT-FEEDBACK AMPLIFIER. Datasheet, Texas Instruments, Rev. C, 2009. [Online]. Available: https://www.ti.com.cn/cn/lit/ds/symlink/ ths3201.pdf?ts=1753861828591&ref_url=https%253A%252F%252Fwww.ti.com.cn%252Fproduct %252Fcn%252FTHS3201.\n[R15] LTC5564: UltraFast™ 7ns Response Time 15GHz RF Power Detector with Comparator. Datasheet, Analog Devices, Inc., Rev. B, 2018. [Online]. Available: https://www.analog.com/ media/en/technical-documentation/data-sheets/5564fc.pdf\n[R16] HMC326MS8G: GaAs InGaP HBT MMIC DRIVER AMPLIFIER. Datasheet, Analog Devices, Inc., Rev. 0, 2013. [Online]. Available: https://www.analog.com/media/en/technical-documentation/ data-sheets/hmc326.pdf\n[R17] Zhou T, et al. Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit. Nat Photonics 15, 367-373 (2021).\n[R18] Liu C, et al. A programmable diffractive deep neural network based on a digital-coding metasurface array. Nat Electron 5, 113–122 (2022).\n[R19] Song A, Murty Kottapalli SN, Goyal R, Schölkopf B, Fischer P. Low-power scalable multilayer optoelectronic neural networks enabled with incoherent light. Nat Commun 15, (2024).\n[R20] Gao X, et al. Programmable surface plasmonic neural networks for microwave detection and processing. Nat Electron 6, 319-328 (2023).\n[R21] Song A, Murty Kottapalli SN, Goyal R, Schölkopf B, Fischer P. Low-power scalable multilayer optoelectronic neural networks enabled with incoherent light. Nat Commun 15, (2024).\n[R22] Zhou T, et al. Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit. Nat Photonics 15, 367-373 (2021)..\n[R23] Wang T, et al. Image sensing with multilayer nonlinear optical neural networks. Nat Photonics 17, 408-415 (2023).\n[R24] Gao X, et al. Programmable surface plasmonic neural networks for microwave detection and processing. Nat Electron 6, 319-328 (2023).\n[R25] Bandyopadhyay S, et al. Single-chip photonic deep neural network with forward-only training. Nat Photonics 18, 1335-1343 (2024).\n[R26] Wu T, Li Y, Ge L, Feng L. Field-programmable photonic nonlinearity. Nat Photonics, (2025).\n[R27] Huang Z, Shi W, Wu S, Wang Y, Yang S, Chen H. Pre-sensor computing with compact multilayer optical neural network. Sci Adv 10, (2024).\n[R28] Zhou T, et al. Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit. Nat Photonics 15, 367-373 (2021).\n[R29] Bandyopadhyay S, et al. Single-chip photonic deep neural network with forward-only training. Nat Photonics 18, 1335-1343 (2024).\n[R30] Xia F, et al. Nonlinear optical encoding enabled by recurrent linear scattering. Nat Photonics 18, 1067-1075 (2024).\n[R31]Song A, Murty Kottapalli SN, Goyal R, Schölkopf B, Fischer P. Low-power scalable multilayer optoelectronic neural networks enabled with incoherent light. Nat Commun 15, (2024).\n[R32] Ou S, et al. Hypermultiplexed integrated photonics–based optical tensor processor. Sci Adv 11, eadu0228.\n[R33] Zhou T, et al. Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit. Nat Photonics 15, 367-373 (2021).\n[R34] Bandyopadhyay S, et al. Single-chip photonic deep neural network with forward-only training. Nat Photonics 18, 1335-1343 (2024).\n[R35] Xia F, et al. Nonlinear optical encoding enabled by recurrent linear scattering. Nat Photonics 18, 1067-1075 (2024).\n[R36]Song A, Murty Kottapalli SN, Goyal R, Schölkopf B, Fischer P. Low-power scalable multilayer optoelectronic neural networks enabled with incoherent light. Nat Commun 15, (2024).\n[R37] Ou S, et al. Hypermultiplexed integrated photonics–based optical tensor processor. Sci Adv 11, eadu0228.\n[R38] Hu L, et al. A 110-170 GHz Wideband LNA Design Using the InP Technology for Terahertz Communication Applications. Micromachines 14, (2023).\n[R39] Singh SP, Rahkonen T, Leinonen ME, Parssinen A. A 290 GHz Low Noise Amplifier Operating above $f_{max}/2$ in 130 nm SiGe Technology for Sub-THz/THz Receivers. In: 2021 IEEE Radio Frequency Integrated Circuits Symposium (RFIC)) (2021).\n[R40] Singh SP, Rahkonen T, Leinonen ME, Parssinen A. Design Aspects of Single-Ended and Differential SiGe Low-Noise Amplifiers Operating Above fmax/2in Sub-THz/THz Frequencies. IEEE J Solid-State Circuits 58, 2478-2488 (2023).\n[R41] Wang L, Shen Y, Qian Y, Ding Y, Hu S. A 220-GHz LNA With 9.7-dB Noise Figure and 24.6- dB Gain in 40-nm Bulk CMOS. IEEE Trans Circuits Syst II Express Briefs 72, 113-117 (2025).\n[R42] Ji DF, Niu B, Tao HQ, Chen TS, Wang WB. A D Band Zero Bias Detector Chip Using Schottky Diode. In: 2022 Photonics & Electromagnetics Research Symposium (PIERS)) (2022).\n[R43] Ludwig F, et al. Graphene field-effect transistors as THz detectors: Distinguishing between resistive self-mixing and the hot-carrier thermoelectric effect. In: 2023 48th International Conference on Infrared, Millimeter, and Terahertz Waves (IRMMW-THz)) (2023).\n[R44] Wang L, Fei X, Tang XH, Wu SY. A fast-speed pulse detector based on Si-Schottky diode. In: 2008 Asia-Pacific Microwave Conference) (2008).\n[R45] Haque MM, Chowdhury MIB. Effects of the field-dependence of the carrier mobility on the base transit time of SiGe HBTs having non-uniformly doped base. In: 2015 International Conference on Advances in Electrical Engineering (ICAEE)) (2015).\n[R46] Yang SJ, et al. Ultrafast 27 GHz cutoff frequency in vertical WSe2 Schottky diodes with extremely low contact resistance. Nat Commun 11, (2020)."}, {"round": 2, "reviewer_comments": {"reviewer_1": "I would like to thank the authors for responding to my comments. However, there are some points that remain to be clarified:\n1- There are different values for latency which I found confusing:\n-) 4.5 ns nonlinearity latency in the original manuscript. As I understand, that does not reflect the nonlinearity latency and was mentioned by mistake. The actual nonlinearity latency was 2.3 us. Is that correct? If so, total latency (with low speed electronics) would have been 3*2.3 + 2.407 = 9.307us. Please clarify.\n-) 2.48 μs system latency and 17.7 ns nonlinearity latency in the revised manuscript. These seem to be based on new measurements on a complete set of new RF components (high-speed). Basically, a new version of the system has been resubmitted but full system classification measurements have not been repeated. Is that correct?\n2- Related to the pervious point: On page 13 of the rebuttal, the authors mention: “This functional equivalence ensures our previously demonstrated performance enhancements (in image classification and in action recognition) remain valid, while achieving a faster response speed”. While generally this can be true, to avoid confusion I have the following comment:\nAs I understand, full system performance measurements have not been repeated with the high-speed version of the system and test boards are fabricated to show faster nonlinearity separately. Is that the case? If so, which is very useful addition to the paper, this should be clarified. The original lower speed components are removed from the revised supplementary document. If full system performance measurements (i.e. classification experiments) have not been redone, the authors should provide the list of both sets of components and their corresponding results. The total system latency and power consumption for both original and new RF components should be provided. This way the reader will fully understand the system capabilities and what has been actually demonstrated.\nTo summarize, I think it is OK if the authors have not repeated the classification measurement with the high-speed components as long as the they clearly mention that and include all the relevant information for low and high speed results.", "reviewer_2": "The authors have clearly implemented detailed modifications, and I appreciate the effort they put into the manuscript. A number of minor questions and points of clarification remain regarding methodological details and aspects of the experimental validation to fully demonstrate the robustness and generalizability of the proposed approach. I therefore encourage the authors to provide additional explanations and supporting evidence, which would significantly strengthen the clarity, rigor, and overall impact of the manuscript. In essence I believe the work is appropriate for Nature Communications, given the several following modifications are implemented. Below are my detailed comments, together with responses to the changes authors made earlier.\n1- I find the authors’ addition of the Fashion-MNIST classification accuracy directly in the introduction to be appropriate and satisfactory. By explicitly reporting the result alongside the MNIST accuracy, the introduction now provides a clearer and more balanced overview of the model’s performance across both benchmark datasets. In this respect, the authors’ modification sufficiently resolves the concern raised, and the manuscript is improved accordingly. This addition is acceptable.\n2- By including examples where both linear and nonlinear networks correctly classify inputs, Figs. 3 and 4 now provide a balanced and accurate representation of performance. This clarifies that the linear network can achieve correct classifications, consistent with its reported accuracies, while the nonlinear network still demonstrates superior class separability. The added explanations and supplementary data further enhance transparency and the completeness of the results.\n3- I thank the authors for their detailed response and examples, which are informative. I noted that in the “Action Taken” section, only the accuracies are reported. It should be explicitly stated that the improved performance is not due to increased trainable parameters—the nonlinear layers contribute only a negligible number of parameters. The comparison with linear networks of matched parameter counts confirms that the gains stem mainly from the nonlinearity itself. Including this clarification would make the analysis more complete.\n4- The authors’ additional experiments and clarifications regarding the effects of down sampling and binarization on classification accuracy are comprehensive and sufficiently address the concern initially raised. The comparative analyses across MNIST and Fashion-MNIST datasets demonstrate the trade-offs involved and highlight that while down sampling has relatively minor consequences, binarization introduces a more notable degradation in recognition performance. The explicit inclusion of these results in the revised supplementary materials, along with the discussion of future directions involving DAC-based continuous grayscale inputs, represents a valuable enhancement to the manuscript. In this regard, the results are acceptable, and the corrections made to the article are appropriate.\nOn the other hand, several important points remain. First, if multi-bit recognition is to be performed in future implementations, the proposed system should be analyzed in greater detail, particularly with respect to its processing speed. It would be highly informative if the authors could provide an explicit analysis on the expected operational speed of the system with a DAC. Specifically, what is the processing speed of each layer in terms of bits per second?\nThe performance figures reported in the manuscript prompt a methodological question regarding network initialization. Typically, with randomly initialized cascaded layers, one would expect initial accuracies that are inversely proportional to the number of layers, as the network is essentially randomly guessing. (e.g., ~10% for MNIST and Fashion). However, the authors report initial accuracies of 84–85% on MNIST and 70–75% on Fashion-MNIST, which are surprisingly high. Could the authors clarify how the layers are initialized, or if this has any positive influence on the relatively high initial accuracies? If a particular weight initialization strategy is employed, it should be explicitly described in the manuscript, as it may significantly affect reproducibility and understanding of the reported performance.\n5- By systematically benchmarking their MN-DNN architecture against fully connected networks (FCN) and convolutional neural networks (CNN) of comparable scale, the authors convincingly demonstrate that the performance reduction on Fashion-MNIST. The comparative results show that MN-DNN performs on par with or better than parameter-matched nonlinear FCNs, while maintaining competitive accuracy with CNNs. The results presented are satisfactory, and the inclusion of the detailed comparative analysis in the supplementary material is appropriate and appreciated.\n6- While the reported total system delay of 2.48 μs (corresponding to a refresh rate of 403 kHz) demonstrates fast operation, it is important to clarify the concept of real-time performance. Real-time processing is not solely determined by achieving or exceeding a particular frequency threshold (e.g., 30 Hz). Instead, it is defined by the system’s ability to respond to inputs within a bounded and application-specific time window. The latency must consistently satisfy the timing constraints dictated by the intended use case. Although the presented latency values indicate that the system operates well within typical real-time bounds, a discussion explicitly linking these delays to the specific timing requirements of the application would provide a more precise justification of real-time capability. Additionally, if references are cited in the text claiming that systems exceeding 30 Hz qualify as real-time, these should be explicitly indicated to clarify the basis for such assertions.\n7- The clarification of power consumption at both unit-cell (429 mW) and system (252.352 W) levels is appreciated. Inclusion of breakdowns, potential reductions using low-power components, and scalability analysis are all performed properly, I have no further concerns.\n8- The authors have provided a thorough and well-structured clarification regarding the number of trainable parameters in both the linear and nonlinear components of their proposed model.\n9- The manuscript still does not include a clear mention of the train–test split ratio, both in the main text and in the supplementary materials. In particular, the number of training samples utilized remains unspecified, which leaves uncertainty regarding the total dataset size employed in the study. Explicitly reporting this information is crucial for evaluating the validity, reproducibility, and generalizability of the findings.\n10- When displaying the pre-computed dataset in both Fashion-MNIST and MNIST using the t-SNE visualization, it is evident that the outputs for the linear and nonlinear networks differ considerably. I have a minor suggestion regarding these plots. The color maps used for the linear and nonlinear networks is inconsistent. In particular, the linear layer visualization is presented in grayscale, which significantly reduces readability and makes it more challenging to interpret the class structure. Furthermore, in the Supplementary, the linear layer’s t-SNE plots are displayed using colored plots, whereas in the main text they appear in grayscale. This inconsistency in visualization design detracts from the overall clarity of the work, as if there is a difference between the plots that is not immediately obvious. I therefore strongly recommend revising the figures to employ a consistent and easily interpretable color scheme across both the main text and supplementary sections.", "reviewer_3": "I co-reviewed this manuscript with one of the reviewers who provided the listed reports. This is part of the Nature Communications initiative to facilitate training in peer review and to provide appropriate recognition for Early Career Researchers who co-review manuscripts.", "reviewer_4": "The authors have addressed all the comments in my last report. I am happy to recommend its publication in Nature Communications."}, "author_response": "Thank you very much for your valuable suggestions. We have carefully addressed your questions point by point. Please refer to the details as below.\nComment 1.1:\n1- There are different values for latency which I found confusing:\n-) 4.5 ns nonlinearity latency in the original manuscript. As I understand, that does not reflect the nonlinearity latency and was mentioned by mistake. The actual nonlinearity latency was 2.3 us. Is that correct? If so, total latency (with low speed electronics) would have been 3*2.3 + 2.407 = 9.307μs. Please clarify.\n-) 2.48 μs system latency and 17.7 ns nonlinearity latency in the revised manuscript. These seem to be based on new measurements on a complete set of new RF components (high-speed). Basically, a new version of the system has been resubmitted but full system classification measurements have not been repeated. Is that correct?\nResponse:\nThank you for your insightful question. Your understanding is entirely correct. In the original manuscript, the nonlinear metasurface employs low-speed components (RF amplifier RFFM4527, RF detector LMH2110 and operational amplifier TLV9001). This configuration results in a single-layer nonlinear delay of 2.3 μs and a total system delay of approximately 9.307 μs.\nIn the first revision, we retain the basic architecture of the nonlinear unit (energy detection and gain control) but replace the components with high-speed RF components, including the RF amplifier HMC326, RF detector LTC5536, and operational amplifier THS3201. As a result, the single-layer nonlinear delay is reduced to 17.7 ns, and the total system delay is reduced to 2.48 μs.\nComment 1.2:\nRelated to the pervious point: On page 13 of the rebuttal, the authors mention: “This functional equivalence ensures our previously demonstrated performance enhancements (in image classification and in action recognition) remain valid, while achieving a faster response speed”. While generally this can be true, to avoid confusion I have the following comment:\nAs I understand, full system performance measurements have not been repeated with the high-speed version of the system and test boards are fabricated to show faster nonlinearity separately. Is that the case? If so, which is very useful addition to the paper, this should be clarified. The original lower speed components are removed from the revised supplementary document. If full system performance measurements (i.e. classification experiments) have not been redone, the authors should provide the list of both sets of components and their corresponding results. The total system latency and power consumption for both original and new RF components should be provided. This way the reader will fully understand the system capabilities and what has been actually demonstrated.\nTo summarize, I think it is OK if the authors have not repeated the classification measurement with the high-speed components as long as the they clearly mention that and include all the relevant information for low and high speed results.\nResponse:\nThank you for your insightful and constructive comments. We sincerely appreciate your kind understanding and supportive suggestion that “It is OK if the authors have not repeated the classification measurement with the high-speed components as long as the they clearly mention that and include all the relevant information for low and high speed results.”\nTo enhance the completeness of our work and facilitate reader understanding, we took the initiative to conduct full system performance measurements in this revision. All experiments were carried out on the latest version of the high-speed multilayer nonlinear metasurface (as shown in Figs. R1 and R3). Specifically, we repeated all classification tasks (MNIST, Fashion-MNIST, and posture classification) using the updated high-speed nonlinear network.\nThe experimental results demonstrate the following:\n Experimental results show that the nonlinear metasurface integrated with high-speed components can fully replace its low-speed counterpart, successfully implementing a ReLU- like activation function with tunable thresholds and slopes.\n Classification experiments further demonstrate that systems integrating high-speed components maintain classification accuracy comparable to low-speed systems in tasks such as image classification and posture recognition, while significantly improving response speed.\n Moreover, under identical input image conditions, the output field distributions of high- speed systems closely match those of low-speed systems, further confirming their functional equivalence in nonlinear operations.\nWe believe that the additional experiments effectively demonstrate the capability of the nonlinear system employing high-speed devices in classification tasks. The system achieves significantly reduced latency while retaining high classification accuracy. These results also help prevent potential confusion caused by evaluating multiple systems with different delays, ensuring consistent interpretation.\nThe specific responses include the following four aspects:\n(1) Nonlinear metasurface with high-speed components: samples and characterization\n(2) MNIST classification using the high-speed system\n(3) Fashion-MNIST classification using the high-speed system\n(4) Posture recognition using the high-speed system\n1. Nonlinear metasurface with high-speed components: samples and characterization\nWe have fabricated three nonlinear metasurfaces, each integrated with high-speed components, for applications in nonlinear diffractive neural networks. Each metasurface comprises a 14×14 array of units, as illustrated in Figure R1. Specifically, each unit features an LTC5564 RF detector on the front side, while the back side is equipped with an HMC326 RF amplifier and a THS3201 operational amplifier.\nFigure R1. Photograph of the nonlinear metasurfaces integrated with high-speed components. a. Front and back views of a single nonlinear unit, showing the high-speed RF detector LTC5564 on the front side, and the high-speed RF amplifier HMC326 together with the high-speed operational amplifier THS3201 on the reverse side. b-d. Photographs of three high-speed nonlinear metasurfaces, which function as nonlinear layer 1, layer 2, and layer 3, respectively, in the nonlinear diffractive neural network.\nThe transmission coefficient of the nonlinear metasurface was tested within a focused lens antenna system with varying incident intensities, as illustrated in Figure R2a (detailed information of the test system see Supplementary Note 3). By adjusting the incident wave intensity, we measure the transmission coefficient S of the high-speed nonlinear metasurface under different input power 21 levels. A DC power supply is used to provide supply V and control V , enabling the measurement CC BIAS of transmission coefficients under various DC conditions.\nAs illustrated in Figure R2b, no transmitted energy is detected when the incident power on the metasurface remains below the threshold of 1 mW. Once the input power exceeds this value, the output exhibits a linear increase, effectively mimicking the characteristic behavior of a ReLU activation function in neural networks. This measurement was conducted under a fixed configuration with V set to 0.5 V and V to 5 V. Furthermore, the activation threshold can be dynamically BIAS CC adjusted by the bias voltage, as demonstrated in Figure R2c, where a higher V is shown to reduce BIAS the threshold power. In addition, Figure R2d reveals that increasing V enhances the amplifier gain CC and the transmission coefficient of the metasurface unit. This results in a steeper slope of the nonlinear activation function.\nThe test results show that the nonlinear metasurface integrated with high-speed components can achieve ReLU-like nonlinear activation, with its threshold and slope being flexibly controlled by the DC voltage. While maintaining reconfigurability comparable to that of low-speed component schemes, this structure also offers the advantage of high-speed response.\nFigure R2. Nonlinear metasurface testing system and testing results. a. Photograph of the nonlinear metasurface testing system. Radiofrequency (RF) energy of varying intensities is transmitted from the VNA's port 1 through a coaxial cable to the top-emission antenna. After passing through the nonlinear metasurface prototype, the transmitted wave reaches the bottom reception antenna and returns to the VNA's port 2. b. With V = 1V and V BIAS CC = 5V, the nonlinear metasurface unit exhibits zero transmission for the incident energies below 1 mW and a linear increase above this threshold, akin to a ReLU function. c. The unit’s nonlinear activation threshold is inversely related to V . d. The unit’s gain is directly proportional to V . BIAS CC\n2. MNIST classification using the high-speed system\nWe replace three low-speed nonlinear metasurfaces in the nonlinear network with three high-speed ones for classification experiments. Apart from substituting these three nonlinear metasurfaces, all other settings in the testing system remain unchanged. The constructed high-speed nonlinear network and testing system are shown in Figure R3.\nFigure R3. Photographs of the high-speed nonlinear network and experimental setup. a. Photograph of the measurement scenario, where a horn antenna illuminates the input layer, directing electromagnetic waves carrying image information into the high-speed nonlinear network. The output field distribution is scanned using a two-dimensional near-field scanner. b. Side view of the high-speed nonlinear network, in which all three original low-speed nonlinear metasurfaces have been replaced by high-speed nonlinear metasurfaces. c. Photograph of a high-speed nonlinear metasurface. d. Back view of the high-speed nonlinear network.\nOn the MNIST dataset, we test classification accuracy using 5000 random handwritten digits (see Supplementary Note 13 for the rapid detection method). The nonlinear network with high-speed components achieves 92.6% classification accuracy, matching the 92.4% accuracy of the low-speed version and significantly exceeding the linear network’s 88.5% by a margin of 4.1%. These testing results confirm the superior performance of the high-speed nonlinear network. As shown in Figure R4, the output field for digit “4” concentrates strongly in the corresponding region with highest energy, correctly classifying it. Similar results are observed for other images, supporting the effective classification capability of the high-speed nonlinear network.\nFigure R4. MNIST testing results using the high-speed nonlinear network. a. Input image. b. Output field distribution of the high-speed nonlinear network. c. Energy distribution across ten output regions. The network focuses energy on correct regions, enabling accurate classification.\nWhen processing complex images, the high-speed nonlinear network demonstrates significant advantages compared with linear network. As shown in Figure R5, for a digit “4” sample often mistaken for “9”, the output energy of the high-speed nonlinear network is correctly concentrated in the “4” region. In contrast, the linear network incorrectly produces peak energy in the “9” region, leading to misclassification. Similar behavior is observed in other complex image classification tasks, confirming the superiority of the proposed nonlinear network in handling challenging images. More test results are provided in Supplementary Note 14.\nFigure R5. The testing results of MNIST dataset on linear and high-speed nonlinear networks. Comparative output field distributions for images from nonlinear and linear diffraction neural networks, with the nonlinear network focusing on correct regions and the linear network on incorrect ones. Energy distribution across ten regions verifies the nonlinear network's accurate classification versus the linear network’s inaccuracies.\nCompared to the nonlinear network based on low-speed components, high-speed one exhibits similar output field distributions. As shown in Figure R6 for the digit “5”, both systems concentrate the maximum field values in the target region, with over 60% of the energy located there. Minor energy differences in other regions (e.g., near the “3”) are negligible and do not impact classification. These slight variations may result from assembly or device inconsistencies. Additional image tests confirm consistent behavior in peak location and energy profile, indicating that the high-speed nonlinear metasurface can effectively replace the low-speed one in implementing the nonlinear activation function.\nFigure R6. The testing results of MNIST dataset on high-speed and low-speed (original) nonlinear networks. A comparison of the output field distributions between the high-speed and low-speed nonlinear networks shows that both focus energy on similar regions. The energy distributions across the ten regions are also comparable.\n3. Fashion-MNIST classification using the high-speed system\nOn the Fashion-MNIST dataset, we test classification accuracy using 5000 random images (see Supplementary Note 13 for the rapid detection method). The nonlinear network with high-speed components achieves 77.4% classification accuracy, matching the 77.0% accuracy of the low-speed version and significantly exceeding the linear network’s 72.5% by a margin of 4.9%. These testing results confirm the superior performance of the high-speed nonlinear network. As shown in Figure R7, the output field for “Sandal” concentrates strongly in the corresponding region with highest energy, correctly classifying it. Similar results are observed for other images, supporting the effective classification capability of the high-speed nonlinear network.\nFigure R7. Fashion-MNIST testing results using the high-speed nonlinear network. a. Input image. b. Output field distribution. c Energy distribution across ten output regions. The network focuses energy on correct regions, enabling accurate classification.\nWhen processing complex images, the high-speed nonlinear network demonstrates significant advantages compared to the linear network. As shown in Figure R8, for an image of a “Sandal” easily mistaken for an “Ankle boot”, the output energy of the high-speed nonlinear network is correctly concentrated in the “Sandal” region. In contrast, the linear network incorrectly produces peak energy in the “Ankle boot” region, leading to misclassification. Similar behavior is observed in other complex image classification tasks, confirming the superiority of the proposed nonlinear network in handling challenging images. Additional test results are provided in Supplementary Note 33.\nFigure R8. The testing results of Fashion-MNIST dataset on linear and high-speed nonlinear networks. Comparative output field distributions for images from nonlinear and linear diffraction neural networks, with the nonlinear network focusing on correct regions and the linear network on incorrect ones. Energy distribution across ten regions verifies the nonlinear network's accurate classification versus the linear network’s inaccuracies.\nCompared to the nonlinear network based on low-speed components, high-speed one exhibits similar output field distributions. As shown in Figure R9 for the image “Sandal”, both systems concentrate the maximum field values in the target region, with over 60% of the energy located there. Minor energy differences in other regions (e.g., near the “Bag”) are negligible and do not impact classification. These slight variations may result from assembly or device inconsistencies. Additional image tests confirm consistent behavior in peak location and energy profile, indicating that the high-speed nonlinear metasurface can effectively replace the low-speed one in implementing the nonlinear activation function.\nFigure R9. The testing results of Fashion-MNIST dataset on high-speed and low-speed (original) nonlinear networks. A comparison of the output field distributions between the high-speed and low-speed nonlinear networks shows that both focus energy on similar regions. The energy distributions across the ten regions are also comparable.\n4. Posture recognition using the high-speed system\nWe repeat the posture recognition measurement using the high-speed nonlinear network, as shown in Figure R10. The measurement system remains identical to the original (low-speed) one, except that three low-speed nonlinear metasurfaces are replaced with high-speed ones. Test results demonstrate an accuracy of 92.5%, which is comparable to the simulation result of 93.06%. This confirms the effectiveness of the high-speed system in posture recognition tasks.\nOn the output plane, the output voltages from eight RF detectors are logged to measure the intensity across eight distinct regions. Classification is determined by the posture corresponding to the detector with the highest voltage. Test results are shown in Figure R11, where the mean output voltage of the detectors is represented by bar heights and their variances by error bar lengths. For instance, during the \"Scratching Head\" motion, the corresponding detector records an output voltage nearing 1.2 V, while all others remain below 0.5 V, which indicates that it has the highest voltage, thus classifying the posture as “Scratching Head”. These results validate that the proposed high-speed nonlinear network can effectively classify postures.\nFigure R10. Posture recognition using the high-speed nonlinear network.\nFigure R11. Posture recognition results. Experimental results from 50 repetitions of eight postures, showing the accurate classification as confirmed by voltage readings across the eight output regions.\nAction Taken:\nTo ensure overall consistency, the revised manuscript features updated unit test results from the high- speed nonlinear metasurface, along with revised classification measurements all based on the high- speed system's results.\n[Architecture of nonlinear metasurface unit]\nFig. 2 | Nonlinear metasurface unit configuration. a The nonlinear metasurface unit, featuring four metal and three dielectric layers, with F4B as the top and bottom dielectrics and FR4 as the middle layer. The unit functions as a summation, nonlinear activation, and amplitude modulation device. b Front and back views of the nonlinear metasurface unit, featuring an octagonal receive antenna on the front with an integrated RF detector, and on the back, a corresponding octagonal transmit antenna accompanied by an RF amplifier and voltage adder. c Operational diagram of the nonlinear metasurface unit, showing the receive antenna directing EM energy to the back for amplification and a portion to the RF detector for rectification. The rectified voltage and an external bias voltage can modulate the RF amplifier’s DC control voltage V , with V controlling the gain. The amplified RF energy CTRL CC is then radiated by the transmit antenna. d The output DC voltage of the RF detector is positively correlated with the incident power density on the nonlinear metasurface. e The gain of the RF amplifier is jointly determined by DC voltages V and V . f The RF amplifier’s gain exhibits a minimal value at low V , effectively cuts off, and CC CTRL CC subsequently increases to a saturation point with further V increments. g The RF amplifier’s gain is minimal at CC low V , effectively cuts off, and then rises to a plateau as the additional V increases. h For V > 1.2V, the CTRL CTRL BIAS nonlinear metasurface unit is operated linearly, with the output energy directly proportional to the input energy, and gain escalating with V . i The transmission coefficient of the nonlinear metasurface unit rises from a minimal value CC at low incident intensities to a plateau at higher levels, with V reducing the energy threshold for transition from BIAS the cutoff to transmission. j The transmission coefficient of the nonlinear metasurface unit escalates with the incident energy, reaching a plateau, and is enhanced by increased V at constant input energy levels. k With V = 1V and CC BIAS V = 5V, the nonlinear metasurface unit exhibits zero transmission for the incident energies below 1 mW and a CC linear increase above this threshold, akin to a ReLU function. l The unit’s nonlinear activation threshold is inversely related to V . m The unit’s gain is directly proportional to V .\nBIAS CC\n[Handwritten digits classification by MN-DNN]\nFig. 3 | Handwritten digits classification. a Illustration of the MNIST dataset for recognition, including 10 classes of handwritten digits (0-9). b Schematic of the input image encoding process, where binarized images are translated into transmission coefficients of the input layer metasurface. c Photo of the testing scenario, where a horn antenna illuminates the input layer, directing the EM waves carrying the image information into MN-DNN. The output field distribution is scanned using a two-dimensional near-field scanner. d Comparison of linear and nonlinear networks is facilitated by two optimized designs: a nonlinear network with interleaved 3 linear and 3 nonlinear layers, and a linear network with only 3 linear layers. e t-SNE visualization of test set outputs for both networks, demonstrating that the nonlinear network achieves more distinct class separation and tighter intra-class clustering. f Confusion matrices for 5000 test images, showing 92.6% and 88.5% accuracy of the nonlinear network and linear network. g Sampled output field distributions generated by the linear and nonlinear networks for a simple image, demonstrating that both networks successfully focus on the target region. h Sampled output field distributions for two complex images from two networks, in which the nonlinear network focuses on the correct regions while the linear network focuses on incorrect regions.\n…The experimental results demonstrate that the nonlinear network achieves a classification accuracy of 92.6% on the 5,000 MNIST images (compared to a simulated accuracy of 92.81%), exhibiting a clear advantage over the linear network (88.5% accuracy, Fig. 3f).\n[Supplementary Note 14: Linear and nonlinear network performance on MNIST]\nSupplementary Figure S24 | Performance comparison of linear and nonlinear networks on simple MNIST images. For simple, well-defined images in the MNIST dataset, both linear and nonlinear diffractive neural networks produce output fields focused at the correct positions, enabling accurate classification.\nSupplementary Figure S25 | The testing results of MNIST dataset on linear and nonlinear networks. Comparative output field distributions for three images from nonlinear and linear diffraction neural networks, with the nonlinear network focusing on correct regions and the linear network on incorrect ones. Energy distribution across ten regions verifies the nonlinear network's accurate classification versus the linear network's inaccuracies.\n[Fashion-MNIST classification by MN-DNN]\nFig. 4 | Fashion-MNIST dataset recognition. a Overview of 10-class fashion item dataset for recognition. b Comparison of two network architectures: the nonlinear network with alternating linear and nonlinear layers, and the linear network with only linear layers. c Sampled output field distributions generated by the linear and nonlinear networks for two simple images, demonstrating that both networks successfully focus on the target regions. d Sampled output-field distributions for three complex images from both networks, with the nonlinear network focusing on correct regions and the linear network on incorrect regions. e The recognition accuracy increases with the number of nonlinear layers in the network. f The effect of nonlinear-layer positioning on the accuracy, with the front and back placements outperforming the central insertions. g The impact of activation function on the recognition accuracy, with ReLU showing the highest improvement among the functions tested.\nDuring experimental evaluation, a set of 5,000 randomly sampled Fashion- MNIST test images is intensity-encoded and processed through both nonlinear and linear network configurations. The recorded recognition accuracy reaches 77.4% and 72.5% for the nonlinear and linear networks, respectively.\n[Supplementary Note 18: Linear and nonlinear network performance on Fashion-MNIST]\nSupplementary Figure S32 | Performance comparison of linear and nonlinear networks on simple, well- defined Fashion_MNIST images. For simple, well-defined images in the Fashion_MNIST dataset, both linear and nonlinear diffractive neural networks produce output fields focused at the correct positions, enabling accurate classification.\nSupplementary Figure S33 |The testing results of the Fashion_MNIST dataset on linear and nonlinear networks. Comparative output field distributions for three images from nonlinear and linear diffraction neural networks, with the nonlinear network focusing on correct regions and the linear network on incorrect ones. Energy distribution across ten regions verifies the nonlinear network's accurate classification versus the linear network's inaccuracies.\n[Supplementary Note 15: Error-analysis of simulation and experimental results]\nSupplementary Figure S26 | Experiment and simulation results for MNIST and Fashion-MNIST recognition. a. MNIST simulation and experimental results, which demonstrate a strong correspondence in high-energy output, affirming the recognition feasibility. Minor low-energy discrepancies are likely due to antenna posture variations, edge diffraction of the aluminum support, and meta-atom coupling. b. Results of fashion-MNIST simulation and experimental measurements. The high-energy regions exhibit a similarly strong match between simulated and measured output energy distributions.\n[Static posture recognition by MN-DNN]\nFig. 5 | Posture Recognition. a Schematic of MN-DNN for posture recognition, where the scattered EM fields from the test person are captured by the input layer and processed by MN-DNN to classify the gestures, with the output field distribution indicating the classification results. b The Equivalent diagram of MN-DNN. c Photograph of the posture-recognition experiment, in which the LED color changes on the output plane to indicate the gesture classification. d Schematic of the detection setup on the output plane, with eight zones for posture detection, each with an antenna and detector converting the energy into DC voltages (V -V ). These voltages are processed by ADC 1 8 of MCU, controlling the LED colors based on the maximum voltage channel, and transmitted to a personal computer for real-time display. e Experimental results from 50 repetitions of eight postures, showing the accurate classification as confirmed by voltage readings across the eight output regions.\nComment 2.1:\nI find the authors’ addition of the Fashion-MNIST classification accuracy directly in the introduction to be appropriate and satisfactory. By explicitly reporting the result alongside the MNIST accuracy, the introduction now provides a clearer and more balanced overview of the model’s performance across both benchmark datasets. In this respect, the authors’ modification sufficiently resolves the concern raised, and the manuscript is improved accordingly. This addition is acceptable.\nResponse:\nThank you very much for your positive comment.\nComment 2.2:\nBy including examples where both linear and nonlinear networks correctly classify inputs, Figs. 3 and 4 now provide a balanced and accurate representation of performance. This clarifies that the linear network can achieve correct classifications, consistent with its reported accuracies, while the nonlinear network still demonstrates superior class separability. The added explanations and supplementary data further enhance transparency and the completeness of the results.\nResponse:\nThank you very much for your positive comment.\nComment 2.3:\nI thank the authors for their detailed response and examples, which are informative. I noted that in the “Action Taken” section, only the accuracies are reported. It should be explicitly stated that the improved performance is not due to increased trainable parameters—the nonlinear layers contribute only a negligible number of parameters. The comparison with linear networks of matched parameter counts confirms that the gains stem mainly from the nonlinearity itself. Including this clarification would make the analysis more complete.\nResponse:\nThank you for your positive feedback and suggestions. We have revised the relevant sections to further clarify the role of nonlinearity itself in improving performance.\nAction Taken:\n[Handwritten digits classification by MN-DNN]\nThe training outcomes indicate that the nonlinear network achieves a recognition accuracy of 92.81%, which is 4.18% improvement over the 88.63% accuracy of the linear network. It is important to note that this improvement is not resulted from the increase of trainable parameters, as each nonlinear layer only adds two trainable parameters (threshold and slope), and three nonlinear layers contribute a total of six additional parameters. For comparison, a linear network with the same number of parameters achieves at most 89% accuracy, further confirming the performance gain from the nonlinear activation functions (see Supplementary Note 11).\n[Fashion-MNIST classification by MN-DNN]\nThe nonlinear diffraction network achieves 78.8% accuracy on Fashion-MNIST, surpassing the linear network (74.43%) by 4.37 percentage points. Similar to the previous case, this improvement is not due to the increased parameter count, as each nonlinear layer only introduces two trainable parameters (the threshold and slope) with totally six additional parameters, which are negligible compared to the model’s 1458 parameters. Under the same parameter conditions, the linear network achieves 75.08% accuracy (Supplementary Note 11), further demonstrating the performance gain from the nonlinear architecture. The accuracy comparison with other networks is given in Supplementary Note 17.\nComment 2.4:\nThe authors’ additional experiments and clarifications regarding the effects of down sampling and binarization on classification accuracy are comprehensive and sufficiently address the concern initially raised. The comparative analyses across MNIST and Fashion-MNIST datasets demonstrate the trade-offs involved and highlight that while down sampling has relatively minor consequences, binarization introduces a more notable degradation in recognition performance. The explicit inclusion of these results in the revised supplementary materials, along with the discussion of future directions involving DAC-based continuous grayscale inputs, represents a valuable enhancement to the manuscript. In this regard, the results are acceptable, and the corrections made to the article are appropriate.\nOn the other hand, several important points remain. First, if multi-bit recognition is to be performed in future implementations, the proposed system should be analyzed in greater detail, particularly with respect to its processing speed. It would be highly informative if the authors could provide an explicit analysis on the expected operational speed of the system with a DAC. Specifically, what is the processing speed of each layer in terms of bits per second?\nResponse:\nThank you for your valuable suggestions. We will address the issues regarding multi-bit identification and network initialization methods point by point.\nQuestion 1: First, if multi-bit recognition is to be performed in future implementations, the proposed system should be analyzed in greater detail, particularly with respect to its processing speed. It would be highly informative if the authors could provide an explicit analysis on the expected operational speed of the system with a DAC. Specifically, what is the processing speed of each layer in terms of bits per second?\nResponse:\nThank you for your insightful suggestions. It should be noted that, if the time during the input and output stages is disregarded, the latency within the diffractive neural network remains the same regardless of whether the input is single-bit or multi-bit. Specifically, it takes 55.53 ns. However, the total end-to-end processing time of the complete system (including input and output) is constrained by the modulation speed of the input layer. Therefore, our response adopts the overall system throughput (encompassing both input and output processes) as the primary performance metric.\nTaking the FPGA (Intel Stratix 10) and DAC (Texas Instruments DAC3484) used in the input layer as an example, we provide a detailed analysis of the operation mechanism of the input layer equipped with the DACs, and report the processing speeds (in bps) for each layer of the system: the input layer operates at 7.47 Gbps, the nonlinear layer at 177.2 Gbps, the linear layer at 93.9 Tbps, and the output layer at 41.7 Mbps. Detailed analysis and calculations are presented below.\nDetailed operation principle of the DAC-equipped system\nThe system employs FPGAs to control multiple multi-channel DAC chips, achieving high-precision output across 196 analog voltage channels. By utilizing high-speed FPGAs and high-performance DACs, the system enables rapid voltage configuration, thereby facilitating fast multi-bit amplitude control at the metasurface input layer. The procedure is shown in Figure R12. The grayscale data of the input image is first mapped to the transmission coefficients of the input metasurface with 14×14 unit. Then these coefficients are converted into corresponding analog voltages for each unit based on a predefined relationship between transmission coefficient and amplifier control voltage, as provided in Supplementary Note 30. The continuous voltages are discretized at 16-bit resolution, generating 196 digital voltage signals, which are grouped and distributed across multiple FPGAs (synchronized via SYNC signals). The FPGAs further partition the signals into 49 groups of control signals (each containing 4 discrete digital voltage values) and transmit them via high-speed LVDS interfaces to 49 quad-channel DAC (DAC3484) chips. Collectively, the 49 DACs generate 196 distinct analog voltage signals, with each DAC producing four individual outputs, which are then precisely applied to the corresponding control ports of the 14×14 metasurface unit array to achieve point-to-point control. This allows 16-bit resolution control over the transmission coefficient of each unit of the input metasurface.\nThe EM wave passes through the input layer with multi-bit amplitude modulation, then propagates through the diffractive neural network composed of three linear layers and three nonlinear layers arranged alternately, and finally reaches the output layer. The output signal is received by ten small antennas, converted into a voltage signal via RF detectors, and then fed into the analog input ports of an Arduino Due R3 MCU for analog-to-digital conversion.\nFigure R12. Operating principle of the input layer with integrated DACs. A multi-bit image is input to the FPGAs, which drive 49 DACs in parallel. Each DAC outputs 4 analog voltages, for a total of 196 channels. These signals control the gain of the metasurface’s 196 units, enabling precise multi-bit amplitude modulation.\nProcessing speed of each layer\nThe overall processing flow of its multi-bit input primarily involves three stages: input layer encoding (where the gain of metasurface units is controlled via FPGA and DAC), propagation of EM waves through the diffraction neural network (composed of three interleaved linear and nonlinear layers), and output layer detection. For each layer, the processing speed, which is defined as the number of effective bits output per second (in bits per second), is evaluated layer by layer.\n(1) Processing speed of the input layer\nThe input metasurface layer utilizes a 14×14 pixel image as input, with each pixel controlled by a 16- bit high-speed DAC (DAC3484). The total output information amounts to 14×14×16 = 3136 bits. The system latency primarily consists of three components: FPGA processing delay, DAC conversion delay, and metasurface unit response delay. The FPGAs drive 49 DACs in parallel via high-speed LVDS interfaces, with a signal transmission delay of approximately 10 ns. The digital-to-analog conversion delay per channel of the DAC3484 is about 400 ns [R1]. The configuration delay of the metasurface unit mainly arises from the response of the RF amplifiers HMC326, which is approximately 10 ns [R2]. The total system latency is about 420 ns, with the DAC conversion delay being the dominant component. Based on this, the processing speed of the input layer can be calculated as 3136 bits / 420 ns ≈ 7.47 Gbps.\n(2) Processing speed of diffractive layers\nIn a diffractive neural network, both the input and output of its three linear and three nonlinear layers are analog values (i.e., the amplitude and phase of electromagnetic waves), rather than digital bit representations. To standardize the computational metric, the total number of bits processed per layer is defined to be consistent with that of the input layer, namely 3136 bits.\nFor a nonlinear layer, the total number of bits processed is 3136 bits, and the processing time is 17.7 ns, resulting in a processing speed of 3136 bits / 17.7 ns = 177.2 Gbps.\nFor a linear layer, which also processes 3136 bits, the processing time is 33.6 ps, leading to a processing speed of 3136 bits / 33.6 ps = 93.9 Tbps.\n(3) Processing speed of output layer\nThe output layer consists of ten small antennas and their corresponding detectors. The voltage signals output by the detectors are acquired through the analog input ports integrated in the Arduino Due R3 MCU, which features a 10-bit ADC resolution. The total number of bits processed is the product of the number of channels and the ADC resolution, i.e., 10 × 10 = 100 bits. The processing time is approximately 2.4 μs (a detailed calculation of the processing time is provided in Supplementary Note 27). Accordingly, the processing speed is calculated as the total number of bits processed divided by the processing time, i.e., 100 bits / 2.4 μs = 41.7 Mbps.\nTo improve the processing speed of output layer, high-speed ADCs can replace the MCU's internal ADCs, while a high-performance FPGA enables rapid signal processing (see Supplementary Note 27). This approach would reduce the output layer latency to 3–5 ns and increases the processing speed to 20 Gbps.\nFigure R13. Schematic of the diffractive neural network integrating DACs, emphasizing the processing-speed hierarchy. a. Network hierarchy, including an input layer with integrated DACs, three alternating linear and nonlinear layers, and an output layer. b. Processing-speed comparison across the layers.\nIn summary, the processing speeds of each layer are as follows: the input layer operates at 7.47 Gbps, the nonlinear layer at 177.2 Gbps, the linear layer at 93.9 Tbps, and the output layer at 41.7 Mbps, as illustrated in Figure R13. Although the input layer's processing speed is limited by the integrated DAC performance and is slower than the linear and nonlinear layers in the diffractive neural network, it operates at the Gbps level, enabling high-speed computation. Employing higher-speed DACs (e.g., TI DAC39J84, ADI AD9172) can effectively further enhance the processing performance of the input layer.\nReferences:\n[R1] DAC3484: Quad-Channel, 16-Bit, 1.25 GSPS Digital-to-Analog Converter (DAC). Datasheet, Texas Instrument, Rev. 0, 2015. [Online]. Available: https://www.ti.com/lit/ds/symlink/dac3484.pdf? ts=1756719342368.\n[R2] HMC326MS8G: GaAs InGaP HBT MMIC DRIVER AMPLIFIER. Datasheet, Analog Devices, Inc., Rev. 0, 2013. [Online]. Available: https://www.analog.com/media/en/technical-documentation/ data-sheets/hmc326.pdf.\nAction Taken:\nThe detailed description of the MN-DNN integrated with DACs, along with the processing speeds of each layer, has been added in Supplementary Note 30: Input layer metasurface and method for multi-bit input.\nQuestion 2: The performance figures reported in the manuscript prompt a methodological question regarding network initialization. Typically, with randomly initialized cascaded layers, one would expect initial accuracies that are inversely proportional to the number of layers, as the network is essentially randomly guessing. (e.g., ~10% for MNIST and Fashion). However, the authors report initial accuracies of 84–85% on MNIST and 70–75% on Fashion-MNIST, which are surprisingly high. Could the authors clarify how the layers are initialized, or if this has any positive influence on the relatively high initial accuracies? If a particular weight initialization strategy is employed, it should be explicitly described in the manuscript, as it may significantly affect reproducibility and understanding of the reported performance.\nResponse and Action Taken:\nDuring the network weight initialization process, we employ a random initialization method. Regarding the issue of the initially reported high accuracy, this was due to a labeling error in the figures where the validation accuracy after the first training epoch (epoch=1) was incorrectly labeled as the initial value (epoch=0). In reality, all validation steps were conducted after the completion of each training epoch, and the initial accuracies (i.e., before any training) were not shown. The reported accuracies of 84%–85% on MNIST and 70%–75% on Fashion-MNIST correspond to the results after the first epoch. We sincerely apologize for the misunderstanding caused by this error. To more comprehensively reflect the model's initial performance, we have supplemented the accuracy at the beginning of training (epoch = 0), which fluctuates approximately between 5% and 30%. For instance, in the case of MNIST recognition, when different types of input images (original MNIST images, down-sampled MNIST images, and images subjected to both down-sampling and binarization) are fed into the nonlinear network, the initial accuracies are 6.25%, 11.82%, and 18.75%, respectively, as shown in Figure R14.\n0.8\n)\n%\n(0.6\ny\nc\na\nr\nu\nc0.4\nc\nA Original Image\nDown Sampling\n0.2\nDown Sampling+ Discretization\n0 5 10 15 20\nEpoch\nFigure R14. A plot of full training accuracy including initial values. For three types of MNIST inputs, the initial accuracies are 6.25%, 11.82%, and 18.75%, respectively.\nWe sincerely apologize for the misunderstanding caused by the mistake. We have thoroughly reviewed and corrected all relevant figures, explicitly adding the initial accuracy at epoch=0 to prevent any misunderstanding. The specific revisions are as follows:\n[Supplementary Note 6: Effect of image preprocessing on recognition accuracy]\nSupplementary Figure S10 |Impact of three types of Fashion_MNIST dataset inputs on the recognition accuracy of a linear network. a. Input images are the original Fashion_MNIST images, achieving a recognition accuracy of 78.48%. b. Input images are down sampled to 14×14 pixels, yielding a recognition accuracy of 76.95%. c. Input images are down sampled to 14×14 pixels and then binarized with a threshold of 0, resulting in a recognition accuracy of 74.43%. d. Performance comparison of network architectures with three different inputs.\nSupplementary Figure S11 |Impact of three types of Fashion_MNIST dataset inputs on the recognition accuracy of a nonlinear network. a. Input images are the original Fashion_MNIST images, achieving a recognition accuracy of 83.71%. b. Input images are down sampled to 14×14 pixels, yielding a recognition accuracy of 82.68%. c. Input images are down sampled to 14×14 pixels and then binarized with a threshold of 0, resulting in a recognition accuracy of 78.80%. d. Performance comparison of network architectures with three different inputs.\nSupplementary Figure S12 |Impact of three types of MNIST dataset inputs on the recognition accuracy of a linear network. a. Input images are the original MNIST images, achieving a recognition accuracy of 91.28%. b. Input images are down sampled to 14×14 pixels, yielding a recognition accuracy of 91.07%. c. Input images are down sampled to 14×14 pixels and then binarized with a threshold of 0, resulting in a recognition accuracy of 88.63%. d. Performance comparison of network architectures with three different inputs.\nSupplementary Figure S13 | Impact of three types of MNIST dataset inputs on the recognition accuracy of a nonlinear network. a. Input images are the original MNIST images, achieving a recognition accuracy of 94.78%. b. Input images are down sampled to 14×14 pixels, yielding a recognition accuracy of 94.68%. c. Input images are down sampled to 14×14 pixels and then binarized with a threshold of 0, resulting in a recognition accuracy of 92.81%. d. Performance comparison of network architectures with three different inputs.\nComment 2.5:\nBy systematically benchmarking their MN-DNN architecture against fully connected networks (FCN) and convolutional neural networks (CNN) of comparable scale, the authors convincingly demonstrate that the performance reduction on Fashion-MNIST. The comparative results show that MN-DNN performs on par with or better than parameter-matched nonlinear FCNs, while maintaining competitive accuracy with CNNs. The results presented are satisfactory, and the inclusion of the detailed comparative analysis in the supplementary material is appropriate and appreciated.\nResponse:\nThank you very much for your positive comment.\nComment 2.6:\nWhile the reported total system delay of 2.48 μs (corresponding to a refresh rate of 403 kHz) demonstrates fast operation, it is important to clarify the concept of real-time performance. Real-time processing is not solely determined by achieving or exceeding a particular frequency threshold (e.g., 30 Hz). Instead, it is defined by the system’s ability to respond to inputs within a bounded and application-specific time window. The latency must consistently satisfy the timing constraints dictated by the intended use case. Although the presented latency values indicate that the system operates well within typical real-time bounds, a discussion explicitly linking these delays to the specific timing requirements of the application would provide a more precise justification of real-time capability. Additionally, if references are cited in the text claiming that systems exceeding 30 Hz qualify as real-time, these should be explicitly indicated to clarify the basis for such assertions.\nResponse:\nThank you for your valuable suggestions. You point out that “Real-time processing is not solely determined by achieving or exceeding a particular frequency threshold (e.g., 30 Hz). Instead, it is defined by the system’s ability to respond to inputs within a bounded and application-specific time window.” We fully agree with this perspective. In response, we have supplemented relevant references and provided a clearer comparison between the delay performance of our system and the timing requirements of specific application scenarios, thereby more rigorously demonstrating its real-time processing capabilities.\nIn posture recognition, real-time requirements vary by application, with no established standard. Existing studies generally agree that such tasks demand processing rates of several tens of fps and millisecond-level delays to achieve basic real-time performance[R1]-[R10]. Our system’s total delay is merely 2.48 μs, which is at least three orders of magnitude lower than typical application requirements, making it well-suited for a wide range of real-time limb movement recognition scenarios.\nSpecifically, in practical applications such as AR, VR, game interaction, and real time video analysis, limb movement recognition imposes diverse performance requirements. For instance, VR and AR applications typically require end to end delays below 20 ms, corresponding to a 50 Hz refresh rate, to preserve user immersion and prevent motion sickness[R1]. More demanding scenarios like boxing motion analysis often require frame rates of at least 60 fps to accurately capture rapid movements[R2]. By comparison, widely used systems such as Kinect V1 and V2 operate at 30 fps[R3]-[R5], representing a common real time standard in current limb tracking technologies. It is worth noting that certain applications, such as abnormal behavior detection, can still achieve real time performance even at lower frame rates around 8 fps[R6].\nIn summary, existing studies typically define real time limb movement recognition as operating within a range of 8 to 60 fps, corresponding to delays spanning tens of milliseconds, with specific requirements depending on task and motion characteristics as detailed in Table 1. By comparison, our method achieves a refresh rate of 403 kHz while maintaining a system delay of only a few microseconds. This represents a reduction in delay of at least three orders of magnitude compared to conventional systems, fully satisfying real time processing requirements for this task.\nTable R1. Comparison of real-time performance in representative posture recognition works.\nRepresentative works Applications Frame\nRate\nOnline, Real-time Tracking and Recognition of Human Real-time posture 30fps\nActions[R7] recognition\nReal-time 2D Multi-Person Pose Estimation on CPU: Real-time posture 26 fps\nLightweight OpenPose [R8] recognition\nMulti-Person 3D Pose Estimation in Mobile Edge Real-time posture 40fps\nComputing Devices for Real-Time Applications[R9] recognition\nReal-Time Hand Tracking and Collision Detection for Real-time posture 60fps\nImmersive Mixed-Reality Boxing Training on Apple recognition for AR/VR\nVision Pro[R2]\nEvaluating the Impact of Frame Rate on Video Based Real-time posture 8fps\nHuman Action Recognition[R6] recognition for abnormal\nbehavior\nPlayerOne: Egocentric World Simulator[R10] Real-time posture 8fps\nrecognition for game\ninteraction\nReferences:\n[R1] Intel Corporation. Augmented and Virtual Reality for Service Providers. (Intel Corporation, Santa Clara, CA, USA, 2017). https://networkbuilders.intel.com/docs/networkbuilders/augmented- and-virtual-reality-for-service-providers.pdf.\n[R2] Karelin A, Brazhenko D, Kliukovkin G, Chernenko Y. Real-Time Hand Tracking and Collision Detection for Immersive Mixed-Reality Boxing Training on Apple Vision Pro (2025).\n[R3] Shahroudy A, Liu J, Ng T-T, Wang G. NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)) (2016).\n[R4] Cheng X, Zhang L, Tang Y, Liu Y, Wu H, He J. Real-Time Human Activity Recognition Using Conditionally Parametrized Convolutions on Mobile and Wearable Devices. IEEE Sens J 22, 5889-5901 (2022).\n[R5] Wang B, Huang L, Hoai M. Active Vision for Early Recognition of Human Actions. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)) (2020).\n[R6] Harjanto F, Wang Z, Lu S, Feng DD. Evaluating the impact of frame rate on video based human action recognition. In: Proceedings of the 27th Conference on Image and Vision Computing New Zealand). Association for Computing Machinery (2012).\n[R7] Natarajan P, Nevatia R. Online, Real-time Tracking and Recognition of Human Actions. In: 2008 IEEE Workshop on Motion and video Computing) (2008).\n[R8] Osokin D. Real-time 2D Multi-Person Pose Estimation on CPU: Lightweight OpenPose. In: Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods) (2019).\n[R9] Imtiaz Hossain M, Akhter S, Delowar Hossain M, Seon Hong C, Huh E-N. Multi-Person 3D Pose Estimation in Mobile Edge Computing Devices for Real-Time Applications. In: 2023 International Conference on Information Networking (ICOIN)) (2023).\n[R10] Tu Y, Luo H, Chen X, Bai X, Wang F, Zhao H. PlayerOne: Egocentric World Simulator. arXiv (2025).\nAction Taken:\nIn the revised manuscript, we have supplemented relevant references and provided clearer comparisons between our system's delay performance and the timing requirements of specific application scenarios, thereby offering more rigorous validation of its real time capabilities.\n[Dynamic posture recognition by MN-DNN]\n…For posture recognition tasks, real-time performance typically requires a time delay on the order of milliseconds38-40. The proposed system reduces the latency by at least three orders of magnitude, enabling efficient real-time motion recognition.\n[References]\n38. Natarajan P, Nevatia R. Online, Real-time Tracking and Recognition of Human Actions. In: 2008 IEEE Workshop on Motion and video Computing) (2008).\n39. Harjanto F, Wang Z, Lu S, Feng DD. Evaluating the impact of frame rate on video based human action recognition. In: Proceedings of the 27th Conference on Image and Vision Computing New Zealand). Association for Computing Machinery (2012).\n40. Tu Y, Luo H, Chen X, Bai X, Wang F, Zhao H. PlayerOne: Egocentric World Simulator. arXiv (2025).\nComment 2.7:\nThe clarification of power consumption at both unit-cell (429 mW) and system (252.352 W) levels is appreciated. Inclusion of breakdowns, potential reductions using low-power components, and scalability analysis are all performed properly, I have no further concerns.\nResponse:\nThank you very much for your positive comment.\nComment 2.8:\nThe authors have provided a thorough and well-structured clarification regarding the number of trainable parameters in both the linear and nonlinear components of their proposed model.\nResponse:\nThank you very much for your positive comment.\nComment 2.9:\nThe manuscript still does not include a clear mention of the train–test split ratio, both in the main text and in the supplementary materials. In particular, the number of training samples utilized remains unspecified, which leaves uncertainty regarding the total dataset size employed in the study. Explicitly reporting this information is crucial for evaluating the validity, reproducibility, and generalizability of the findings.\nResponse:\nThank you very much for your valuable suggestion. In this study, we use the complete MNIST and Fashion-MNIST datasets, each comprising 60,000 training samples and 10,000 test samples. The posture recognition dataset comprises 6,800 samples, with 70.5% allocated to the training set, 23.5% to the test set, and 6% to the validation set. Detailed descriptions of the dataset splits have been added to the main text to enhance clarity and ensure the reproducibility of the experiments.\nAction Taken:\n[Handwritten digits classification by MN-DNN]\nAs shown in Fig. 3d, we independently train linear and nonlinear neural networks using the full MNIST dataset (60,000 for training and 10,000 for testing) to evaluate their image classification performance.\n[Fashion-MNIST classification by MN-DNN]\nNext, we compare the classification performance of linear and nonlinear networks on the complete Fashion-MNIST dataset, which consists of 60,000 training samples and 10,000 test samples. This dataset comprises ten apparel categories: T-shirts, trousers, pullovers, dresses, coats, sandals, shirts, sneakers, bags, and ankle boots, as illustrated in Fig. 4a.\n[Static posture recognition by MN-DNN]\nA total of 850 data samples are collected for each posture, resulting in 6,800 samples across eight postures, with corresponding labels ranging from 0 to 7. Among these, 70.5% are allocated to the training set, 23.5% to the test set, and 6% to the validation set (details see Supplementary Note 22).\nComment 2.10:\nWhen displaying the pre-computed dataset in both Fashion-MNIST and MNIST using the t-SNE visualization, it is evident that the outputs for the linear and nonlinear networks differ considerably. I have a minor suggestion regarding these plots. The color maps used for the linear and nonlinear networks is inconsistent. In particular, the linear layer visualization is presented in grayscale, which significantly reduces readability and makes it more challenging to interpret the class structure. Furthermore, in the Supplementary, the linear layer’s t-SNE plots are displayed using colored plots, whereas in the main text they appear in grayscale. This inconsistency in visualization design detracts from the overall clarity of the work, as if there is a difference between the plots that is not immediately obvious. I therefore strongly recommend revising the figures to employ a consistent and easily interpretable color scheme across both the main text and supplementary sections.\nResponse:\nThank you very much for your valuable comments. We have standardized the color schemes accordingly. For MNIST, the t-SNE plots in both the main text and the supplement now use a set of ten highly distinct colors, as shown in Figure R15a. For Fashion-MNIST, we replaced the original gradual Copper colormap in the t-SNE plot with the high-contrast color set to improve category discrimination, as shown in Figure R15b. These changes enhance the visual consistency and improve the discernibility of inter-class structures.\nFigure R15. Updated t-SNE visualization. a. t-SNE projection of the MNIST output field, visualized with a dedicated color scheme (10 colors) for digits 0–9. b. t-SNE projection of the Fashion-MNIST output field, visualized using a different color scheme for its 10 categories.\nAction Taken:\nWe have updated all t-SNE plots for MNIST and Fashion-MNIST in both the main text and the supplementary materials with a consistent high-contrast color set to improve clarity and distinguishability. Example figures from MNIST and Fashion-MNIST are shown here to demonstrate the improved inter-class separation. All other relevant figures have been updated accordingly. Due to the extensive number of images, only a selection is presented here, with comprehensive results provided in the supplementary materials (Supplementary Figure S10-S13, S19, S20, S34-S39, S43 and Extended Figure S3-S6).\n[Main text: Handwritten digits classification by MN-DNN]\nFig. 3 | Handwritten digits classification. a Illustration of the MNIST dataset for recognition, including 10 classes of handwritten digits (0-9). b Schematic of the input image encoding process, where binarized images are translated into transmission coefficients of the input layer metasurface. c Photo of the testing scenario, where a horn antenna illuminates the input layer, directing the EM waves carrying the image information into MN-DNN. The output field distribution is scanned using a two-dimensional near-field scanner. d Comparison of linear and nonlinear networks is facilitated by two optimized designs: a nonlinear network with interleaved 3 linear and 3 nonlinear layers, and a linear network with only 3 linear layers. e t-SNE visualization of test set outputs for both networks, demonstrating that the nonlinear network achieves more distinct class separation and tighter intra-class clustering. f Confusion matrix for 5000 test images, showing 92.4% and 88.5% accuracy of the nonlinear network and linear network. g Sampled output field distributions generated by the linear and nonlinear networks for one simple image, demonstrating that both networks successfully focus on the target region. h Example comparative output field distributions for two complex images from both networks, with the nonlinear network focusing on correct regions and the linear network on incorrect regions.\n[Supplementary Note 11: Comparative analysis with linear networks of equivalent trainable parameters]\nSupplementary Figure S19 | Recognition accuracy of three network architectures on the MNIST dataset. a. A three-layer linear network (trainable parameters: 1,452) achieves a recognition accuracy of 0.8863. b. A novel six-layer linear network with globally consistent linear modulation in the additional three layers (trainable parameters: 1,458) attains an accuracy of 0.889. c. A hybrid network consisting of three linear layers alternated with three nonlinear layers (trainable parameters: 1,458) reaches an accuracy of 0.9281. d. Performance comparison of the three network architectures.\n[Supplementary Note 6: Effect of image preprocessing on recognition accuracy]\nSupplementary Figure S11 |Impact of three types of Fashion_MNIST dataset inputs on the recognition accuracy of a nonlinear network. a. Input images are the original Fashion_MNIST images, achieving a recognition accuracy of 83.71%. b. Input images are down sampled to 14×14 pixels, yielding a recognition accuracy of 82.68%. c. Input images are down sampled to 14×14 pixels and then binarized with a threshold of 0, resulting in a recognition accuracy of 78.80%. d. Performance comparison of the three network architectures.\nREVIEWER #3\nWe sincerely appreciate the reviewer for the time and effort invested in reviewing our manuscript and offering a wealth of insightful comments and thoughtful suggestions. These comments and suggestions are extremely valuable to enhance the quality of our manuscript. Following your constructive guidance, we carefully revised the paper to improve its clarity and comprehensibility.\nREVIEWER #4\nRemarks to the Author:\nThe authors have addressed all the comments in my last report. I am happy to recommend its publication in Nature Communications.\nResponse:\nThank you very much for your positive comment.\nAuthors’ Responses\nREVIEWER #1\nRemarks to the Author:\nI would like to thank the authors for addressing my comments and conducting new measurements.\nThe paper looks good to me and I do not have any further comments. I recommend the paper to be published.\nResponse:\nThank you very much for your positive comment. Your insightful comments have greatly improved the quality of our manuscript.\nREVIEWER #2\nRemarks to the Author:\nThank you for addressing the newly raised concerns by all the reviewers. I believe the implemented changes are satisfactory, and the article is now suitable for publication.\nResponse:\nThank you very much for your positive comment. Your insightful comments have greatly improved the quality of our manuscript.\nREVIEWER #3\nRemarks to the Author:\nI co-reviewed this manuscript with one of the reviewers who provided the listed reports. This is part of the Nature Communications initiative to facilitate training in peer review and to provide appropriate recognition for Early Career Researchers who co-review manuscripts.\nResponse:\nWe sincerely appreciate the time and effort you have invested in reviewing our manuscript and providing a wealth of insightful comments and thoughtful suggestions. These comments are extremely valuable in enhancing the quality of our work."}, {"round": 3, "reviewer_comments": {"reviewer_1": "I would like to thank the authors for addressing my comments and conducting new measurements.\nThe paper looks good to me and I do not have any further comments. I recommend the paper to be published.", "reviewer_2": "Thank you for addressing the newly raised concerns by all the reviewers. I believe the implemented changes are satisfactory, and the article is now suitable for publication.", "reviewer_3": "I co-reviewed this manuscript with one of the reviewers who provided the listed reports. This is part of the Nature Communications initiative to facilitate training in peer review and to provide appropriate recognition for Early Career Researchers who co-review manuscripts."}, "author_response": "We sincerely appreciate the time and effort you have invested in reviewing our manuscript and providing a wealth of insightful comments and thoughtful suggestions. These comments are extremely valuable in enhancing the quality of our work."}]

Decision letter