<figure><img src="image_1.png" /></figure>

HAL
open science

# Dynamic Inverse Design of Broadband Metasurfaces with Synthetical Neural Networks

Yuetian Jia, Zhixiang Fan, Chao Qian, Philipp del Hougne, Hongsheng Chen

► To cite this version:

Yuetian Jia, Zhixiang Fan, Chao Qian, Philipp del Hougne, Hongsheng Chen. Dynamic Inverse Design of Broadband Metasurfaces with Synthetical Neural Networks. Laser and Photonics Reviews, 2024, 18 (10), pp.2400063. 10.1002/lpor.202400063 . hal-04779771

HAL Id: hal-04779771

https://hal.science/hal-04779771v1

Submitted on 13 Nov 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

# Dynamic inverse design of broadband metasurfaces with synthetical neural networks

Yuetian Jia¹,²,³, Zhixiang Fan¹,²,³, Chao Qian¹,²,³,*, Philipp del Hougne⁴,⁵ and Hongsheng Chen¹,²,³,⁵

¹ZJU-UIUC Institute, Interdisciplinary Center for Quantum Information, State Key Laboratory of Extreme Photonics and Instrumentation, Zhejiang University, Hangzhou 310027, China.

²ZJU-Hangzhou Global Science and Technology Innovation Center, Key Lab. of Advanced Micro/Nano Electronic Devices & Smart Systems of Zhejiang, Zhejiang University, Hangzhou 310027, China.

³Jinhua Institute of Zhejiang University, Zhejiang University, Jinhua 321099, China.

⁴Univ Rennes, CNRS, IETR - UMR 6164, F-35000 Rennes, France.

*Corresponding authors: chaoqianzju@zju.edu.cn (C. Qian);
philipp.del-hougne@univ-rennes.fr (P. del Hougne);
hansomchen@zju.edu.cn (H. Chen)

**Abstract:** For over 35 years of research, the debate about the systematic compositionality of neural networks remains unchanged, arguing that existing artificial neural networks are inadequate cognitive models. Recent advancements in deep learning have significantly shaped the landscape of popular domains, however, the systematic combination of previously trained neural networks remains an open challenge. This study presents how to dynamically synthesize a neural network for the design of broadband electromagnetic metasurfaces. The underlying mechanism relies on an assembly network to adaptively integrate pre-trained inherited networks in a transparent manner that corresponds to the metasurface assembly in physical space. This framework is poised to curtail data requirements and augment network flexibility, promising heightened practical utility in complex composition-based tasks. Importantly, we accurately capture the intricate coupling effects between different metasurface segments. We exemplify our approach for two broadband metasurface inverse design problems, reaching accuracies of 96.7% and 95.5%. Along the way, we highlight the importance of suitably formatting the spectral data to capture sharp spectral features. This study marks a significant leap forward in inheriting pre-existing knowledge in neural-network-based inverse design, improving its adaptability for applications involving dynamically evolving tasks.
28

Introduction

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

The endeavor to utilize available labels or observations for formulating decisions in the face of unseen incentives represents a common goal within the realm of deep learning¹. Its success, comparable to a model of the mind, has not materialized overnight. The landscape of artificial intelligence and deep learning has, until now, undergone significant evolution. Recent strides, especially in the domain of deep neural networks, have showcased remarkable progress in replicating human-like reasoning and cognition. Fueled by extensive datasets and advanced architectures, deep learning models have attained exceptional performance across natural language processing², image recognition³, and reinforcement learning⁴. These contemporary models demonstrate the capacity to grasp intricate patterns, recognize hierarchies, and exhibit reasoning capabilities once considered beyond the reach of artificial systems. This transformative journey parallels the integration of machine learning into various fields, notably photonics, where metasurfaces emerge as a prominent representative⁵⁻¹⁸. Metasurfaces, imbued with the power of machine learning, have proven instrumental in optimizing device performance. This has led to the development of highly efficient metasurfaces and plasmonic structures, enabling precise control of light in ways once deemed exceptionally challenging¹⁹⁻²⁸. In the realm of metasurfaces, deep learning assumes a pivotal and indispensable role, serving as the linchpin in the process of uncovering intricate patterns and recognizing hierarchies. This symbiotic relationship empowers metasurfaces with the finesse needed to intricately control the behavior of light. This not only marks a paradigm shift in photonics but also opens up diverse avenues for applications across telecommunications, imaging, sensing, and beyond²⁹⁻⁴¹.

48

49

50

51

52

53

54

55

56

However, amidst these remarkable achievements, it is crucial to acknowledge that a fundamental challenge raised by Fodor and Pylyshyn in 1988 persists – they asserted that artificial neural networks lack the systematic compositionality necessary to be viable models of the human mind⁴². This inherent lack of compositionality further manifests in deep learning models tailored for metasurface design, often resulting in task-specific approaches. Each unique design challenge necessitates a distinct architectural approach, leading researchers to embark on fresh journeys, developing novel networks, and adapting the learning process. This iterative process can be time-consuming and resource-intensive, impeding the swift exploration and optimization of metasurface designs for diverse applications. Furthermore, deep learning models are notorious for their "black-box" nature,
making it arduous to decipher their decision-making processes and comprehend the underlying
physical mechanisms. This lack of interpretability introduces uncertainty in critical applications,
restricting the capacity to derive valuable insights into the fundamental principles governing
metasurface behavior. In addition, another requisite for effective deep learning is the availability of
extensive datasets. Acquiring such vast, well-labeled datasets specific to metasurface behaviors can
be a formidable undertaking, particularly in specialized or niche applications. Often, we must resort
to generating synthetic data or resorting to limited datasets, which may not fully capture the
intricacies of real-world scenarios. Given the limitations, a multi-capacity deep learning framework is
highly desirable, robustly handling diverse metasurface design tasks by adapting knowledge from
related examples to overcome data scarcity, efficiently exploring various functionalities, and
enhancing interpretability through model visualization.

To address the lack of systematic compositionality, we explore a modular approach to metasurface
inverse design with neural networks that inherits knowledge from neural networks previously trained
for the inverse design of different segments of which the new target metasurface is composed. The
resulting synthetical neural network inherits and amalgamates valuable knowledge from prior designs,
akin to passing down essential knowledge from the parent generation to the next generation. This
eco-conscious method capitalizes on pre-existing knowledge to streamline the design process and
significantly alleviate the cost of training the neural network. The synthetical neural network can also
be termed a knowledge-inherited network<sup>43</sup> or a genetic neural network.

The difficulty of inverse-designing metasurfaces with a modular approach that inherits inverse-design
networks for different segments of the metasurface lies in capturing the intricate mutual-coupling
effects between different segments. Given the resonant nature of the metasurface segments, the
combination of reflection spectra of individual segments into that of a composite metasurface is
highly non-trivial. Especially sharp spectral features cannot be predicted intuitively. Previous
preliminary work on knowledge-inherited NNs recently presented in Ref.<sup>43</sup> was limited to the
monochromatic regime and deliberately limited coupling effects as much as possible in order to then
neglect them in the forward model. In contrast, here we tackle the challenging problem of capturing
the broadband coupling effects between and within different metasurface segments. Moreover,
Ref.<sup>43</sup> only explored the regular tiling of metasurface segments in an array-like fashion whereas we
86

consider more intricate segmentations of the final metasurface in the present paper where one
segment is physically surrounded by another, which can be expected to yield more complex coupling
effects.

87

88

89

To illustrate broadband inverse design with a synthetical neural network despite challenging coupling
effects between different metasurface segments, we here design three "parent" components (one
square and one annular) and three corresponding "parent" neural networks that achieve an
impressive average accuracy of 99.6%. These "parent" components and "parent" networks are
compiled into a gene library. For a new "offspring" metasurface, we nest the parent components in
physical space, and then assemble the corresponding "parent" neural networks into an "offspring"
network. Here, we present examples of assembling two and three components from the gene library,
where the accuracy of the "offspring" neural network is recorded at 96.7% and 95.5%, respectively,
forming a sharp contrast with the accuracy of the conventional network at 38.5%. We expect our work
to play a significant role in photonics design by improving its adaptability as well as its frugality with
respect to scarce resources (compute power, time and energy consumption).

90

91

92

93

94

95

96

97

98

99

100

## Results

101

**Notion of synthetical neural network.** In the domain of metasurfaces design and intelligent
application, a central focus resides in the meticulous structural design of meta-atoms, known as
inverse design. This approach is dedicated to crafting structures that meet stringent requirements or
predefined performance criteria in terms how they manipulate electromagnetic waves. The
methodology involves the intricate adjustment of geometric shapes, material properties, or
metasurface distributions to achieve targeted reflective, transmissive, or absorptive characteristics
within designated frequency ranges. In recent years, deep learning has emerged as a prominent
optimization method for this kind of inverse-design task. Artificial neural networks, inspired by the
biological nervous system, consist of interconnected artificial neurons. Each neuron processes input
through weighted mechanisms, contributing to the generated output (Fig. 1). Leveraging the robust
interconnectivity between nodes, neural networks facilitate automated processing and predictions
for complex tasks. However, in conventional deep learning-based metasurface inverse design, the
neural network is typically structured for a specific task. When transitioning to a new task, the pre-
trained neural network, labeled data, and all training outcomes must be discarded. Notably, the

102

103

104

105

106

107

108

109

110

111

112

113

114
disposal of labeled data emphasizes the necessity to reset the learning process entirely, posing a
time-consuming challenge, especially for metasurfaces relying on simulation-generated data. Here,
we seek to overcome this limitation with a synthetical neural network. This method decomposes the
solution space, assigning each split subproblem to its corresponding neural network. Ultimately, an
overarching network assembles all sub-networks. When confronted with an entirely new task, the
selection of suitable neural networks (NNs) and their assembly in a specific order enables the rapid
synthesis of an entirely new synthetical neural network. This is exemplified by the free assembly of
NN 1/2/4 and the addition of a new NN 5 in Fig. 1. Further, due to the drastic reduction in the solution
space, the data required for sub-networks significantly decreases, and the complexity of their
network architecture also decreases. This not only greatly alleviates the difficulty of exploring the
original solution space, but also endows the neural network-based inverse-design process with a
recycling feature, attributed to the flexible assembly characteristics among its sub-networks.

Our modular approach to composing a NN for metasurface inverse design resembles the concept of
transfer learning insofar as it reuses NNs previously trained for related but different tasks. Transfer
learning maintains the architecture of a previously trained NN and retrains its weights for the new
task. Therefore, it does not offer flexibility to combine knowledge learned by multiple NNs previously
trained on related sub-tasks. In contrast, our modular approach allows our synthesized NN to inherit
and combine knowledge from multiple NNs previously trained on related tasks that we consider
relevant based on our understanding of the physical problem. Therefore, we keep the architecture
and weights of the NNs previously trained on various related sub-tasks fixed (we refer to them as
INNs). We integrated these INNs into a synthetic NN composed of an ASNN followed by the INNs. By
training the ASNN we only learn how to combine the inherited knowledge from various INNs. This
methodology makes the NN synthesis transparent and has a clear relation to the metasurface
assembly in the physical reality. By efficiently combining inherited knowledge, our approach strongly
reduces the amount of required training data as well as the computational effort required for training.

**Architecture of synthetical neural network.** For concreteness, we consider the design of a reflective
1-bit coding metasurface that consists of a metallic pattern on top of a dielectric substrate with a
thickness of 2 mm and a relative permittivity of 3.5+0.00245i, backed by a metallic ground plane. The
metallic pattern can be understood as a matrix of elements of size 10 × 10 mm² that are either
metallized or not. The goal of the inverse design is to decide which elements should be metallized for a desired electromagnetic response. As metasurface inverse design is demand-driven, in traditional machine learning-based optimization, the input typically comprises the reflection spectrum ($S_{11}$ parameter), while the output corresponds to the metallic resonant surface (Fig. 2a). When encountering a new task (Target 2), a completely new traditional neural network (TNN 2) is often required. Here, the resonant surface can be discretized into pixel blocks represented as matrix $R_M$, where M represents the number of pixel blocks ($M = 10$ in our case), and each block exhibits a physical size of $p/M$, where the blue/yellow color indicates the absence/presence of metal in that block.

Our optimization strategy takes an unconventional route. Initially, the targeted "offspring" metasurface is partitioned into two "parent" components, a square $R_8$ and one annular ring $R_M^N$ with the ring width of 2 mm, where $M/N$ denote the outer/inner contour side length, respectively. Subsequently, we construct and train inverse design networks for each "parent" component, cataloging them in a "gene" library, each of which can be referred to as an inheritance neural network (INN). The amalgamation of all INNs is then coordinated and deployed by an assembly neural network (ASNN), with the $S_{11}$ parameter of "offspring"/"parent" component representing its inputs/outputs. The overall network composed of the ASNN and the INNs for a specific inverse-design task is referred to as the synthetic neural network. When confronted with a novel task, such as $R_{10}$ in our case, the adaptation process involves the addition of a new INN 2 and a corresponding ASNN (Fig. 2b). This addition facilitates the assembly of two parent neural networks to effectively address the new task, obviating the need for an extensive overhaul of the entire network architecture or the collection of a large volume of $R_{10}$-specific data. It is noteworthy that despite the introduction of new networks (INN 2 and ASNN), each additional INN can be judiciously repurposed, and the structural adjustments required for the ASNN are minimal. This efficiency underscores a reduced investment of time and resources in network design. Furthermore, due to the decomposition of metasurface inverse design into smaller subproblems, the data requirements are substantially diminished, simplifying the problem-solving process for the corresponding inverse-design tasks.

An INN is employed to approximate the relationship between the $S_{11}$ parameters of a given "parent" component and its respective metallic patterns $R$. The $S_{11}$ parameters are significantly
influenced by the metallic structure, often leading to multiple sharp resonant features. To ensure that our INN can properly process the $S_{11}$ spectra, we extract the contour profiles of the $S_{11}$ parameters and resize them into 100 x 100 images, as detailed in the subsequent section. In this configuration, the INN takes a 100 x 100 x 2 $S_{11}$ image as input, where the two channels in the third input dimension represent the real and imaginary components. The output is binarized to yield the corresponding resonant metallic surface in line with the physical 1-bit coding constraint. INN 1/2 generate $R_8$ and $R_{10}^8$, respectively. To tackle the non-uniqueness challenge of inverse-design (multiple resonant surfaces can yield nearly identical $S_{11}$ parameters), the so-far described INN architecture acts as an encoder that is complemented by a decoder. The decoder is a forward design network (FDN) which maps the metasurface $R$ to $S_{11}$ images. Both encoder and decoder are structured as convolutional neural networks (CNNs) and arrange in an encoder-decoder configuration (Fig. 2c). Overall, the encoder-decoder network seeks to approximate the identity function. The loss function quantifies the discrepancy between the input and output $S_{11}$ images. We evaluate the loss function using the mean square error (MSE) metric, in line with established practices in deep learning, and we train the encoder-decoder network using a standard gradient descent algorithm⁴⁴.

Operating as an essential component within the structural assembly phase, the ASNN assumes a critical role in discerning the coupling laws governing interactions among structural constituents. The ASNN's input is a 100 x 100 x 2 $S_{11}$ image. The ASNN's output with dimensions of 100 x 100 x 4 corresponds to two $S_{11}$ images, one for each of the two "parent" components. These two $S_{11}$ images represent the reflection spectra that the two components of which the final metasurface will be composed should yield. Therefore, the ASNN must "understand" the intricate coupling effects between these two components. The two $S_{11}$ images output by the ASNN hence serve as inputs for the INNs. Regarding the ASNN architecture, we heuristically found that it is helpful to structure the ASNN with an encoder and decoder part comprising seven convolutional layers (encoder part) and 6 deconvolutional layers (decoder part). However, unlike the INNs, the ASNN does *not* seek to approximate an identity function. The ASNN is trained with triplet data where each example includes the reflection spectrum of a given metasurface and the reflection spectra of its two constituent components. The loss function is again defined as an MSE that quantifies the discrepancies between the ground truth and the predicted outcomes for the reflection spectra of the two metasurface
components (see Materials and Methods).

**Pre-processing of spectral data.** Due to the inherent physical characteristics of resonant structures, rapid spectral variations typically emerge near resonances, resulting in sharp resonant features that are highly sensitive to minor structural alterations (Fig. 3a). We observed that simply discretizing the spectral data and stacking its real and imaginary components before inputting it into the ASNN is not a suitable data format because it does not yield a satisfactory ASNN performance. This observation makes sense because the ASNN is based on a CNN architecture which is specialized in processing 2D image inputs. Specifically, in Fig. 3b we illustrate the results of training the ASNN with 5,000 samples of the *S*<sub>11</sub> parameter which are formatted as 2002 x 2 arrays, where the first dimensions represent frequency and the second dimension corresponds to real and imaginary parts. Examination of randomly selected test samples highlights a notable discrepancy between predicted (dotted line) and actual values (solid line), particularly near resonances. Evaluation using the Pearson correlation coefficient (PCC) for a 500-test sample dataset yields an accuracy of 55.9%, emphasizing the struggle of the ASNN to precisely predicting resonant features of the *S*<sub>11</sub> parameters with this data format.

To overcome this struggle, we transform the spectral data to an image-like data format that is more suitable for CNNs. Specifically, our approach involves directly extracting the contour of the reflection spectrum and resizing this information into 100 x 100 images, as depicted in Fig. 3a. Harnessing the intrinsic advantages of CNNs in the domain of image processing, we represent the *S*<sub>11</sub> parameters as image inputs, allowing the CNN to efficiently process two-dimensional image data. This method yields a data format from which the CNN-based ASNN can extract more relevant abstract features (Fig. 3c). Importantly, this simple data reformatting process facilitates the learning of relevant features from extensive datasets, eliminating the need for intricate and computationally intensive feature extraction procedures.

**Training results and comparison with conventional neural networks.** We utilize a total of 5,000 samples generated by CST studio suite for normally incident electromagnetic waves in order to train INN 1/2 and ASNN. The very large targeted frequency range is 6-13 GHz. The samples are randomly divided into training, validation, and testing sets at an 8:1:1 ratio. The training results for INNs and ASNN are illustrated in Figs. 4a and 4c, indicating satisfactory validation losses compared to the training losses. The predictions for three arbitrarily selected samples from the test set are depicted in
Figs. 4b and 4d, where the actual and predicted $S_{11}$ images exhibit remarkable similarity, even in the vicinity of sharp spectral features. To quantify the training effectiveness, we employ the structural similarity index (SSIM) as a metric, widely recognized in image processing to assess similarity across luminance, contrast, and structure. Calculating SSIM for the entire test dataset, we obtain results of 98.4% and 99.7% for INN 1/2, and 96.7% for ASNN, providing further quantitative evidence of the robust generalization capabilities of our synthetic neural network.

**Scalability of synthetic neural network.** To further illustrate the adaptable assembly feature and recyclable nature of our network, we embedded an annular ring, $R_{14}^{10}$, within the design of $R_{10}$ (Fig. 5a). For this new $R_{14}$ design task, data from the previous $R_8$ and $R_{10}^8$, along with the networks (INN1/2), could be leveraged. Rebuilding and training the $R_{14}^{10}$ network, akin to $R_{10}^8$, and introducing a new ASNN' were the only requirements. The training outcomes of INN3 with 5,000 examples for $R_{14}^{10}$ and the new ASNN' are depicted in Figs. 5b and 5d, achieving high SSIM values of 99.2% and 95.5%, respectively. The predictions for random samples from the test set are displayed in Figs. 5c and 5e, emphasizing our network's remarkable recyclability in efficiently utilizing resources while consistently delivering high-quality results, and significant knowledge-inheritance capabilities when applied to a new task.

For comparative analysis, we perform the $R_{14}$ design task with the same amount of training data using a CNN, sharing a structure akin to the INN (i.e., featuring both inverse and forward modules). The training results, illustrated in Fig. 5f, reveal an inability of the CNN to converge, showcasing a pronounced underfitting condition with an accuracy of only 70.5%. This substantial underfitting becomes notably evident when examining the $S_{11}$ outcomes for the three test samples in Fig. 5g. The stark inconsistency in results strongly affirms that the CNN is ineffective when exposed to the same training data. This benchmarking further underscores the superiority and efficiency of our synthetic neural network over traditional "indivisible" design methods.

## Discussion

In conclusion, we have introduced the modular broadband synthesis of neural networks for metasurface inverse design that inherits and efficiently combines knowledge from multiple neural networks previously trained on related sub-tasks. Our approach speeds up the metasurface design process by efficiently integrating prior insights and experiences, akin to the genetic inheritance
through generations. Thereby, both the amount of required training data and the required computational resources and compute time can be significantly reduced, such that our approach economizes scarce resources in an environmentally sustainable manner. The proposed low-cost design methodology reduces redundant computations and expedites the design process. Our work demonstrates great adaptability and scalability and, importantly, manages to accurately capture sharp resonant features and intricate coupling effects between metasurface segments. Additionally, the proposed data formatting eliminates the challenges posed by spectral data for CNN-based networks. Looking forward, we anticipate the widespread integration and proliferation of our approach in the field of material science.

## 269 Materials and methods

**Data generation and training details.** The reflection spectrum data of different metasurfaces is generated through CST studio suite, with the incident wave perpendicular at a position 30mm away from the metasurface. In this way, we first generate 5,000 datasets of each "parent" component by randomly setting the metasurface distribution for INN training. Then, for an "offspring" metasurface (assembled by "parent" components), we also follow the similar procedure to generate 5,000 samples for ASNN training. The Adam optimizer⁴⁵ is employed to update the parameters to complete the training of the models. During training, the epochs and batch size of the INN are set to 100 and 64, and those of the ASNN are set to 100 and 128, respectively. The initial learning rates of the INN and the ASNN are both 0.0001, which will continue to shrink by a ratio of 0.5 when the validation loss stops improving, and the update is terminated when the minimum learning rate of $10^{-5}$ is reached. In this way, the optimal model can be obtained quickly and accurately. For the ASNN, the loss function MSE is formally expressed as $MSE = \frac{1}{P*Q} \sum_{i=1}^{P} \sum_{j=1}^{Q} (S_{ij} - S_{ij}')^2$, where $S_{ij}(S_{ij}')$ denotes the jth sampling point of the ground-truth (predicted) $S_{ij}$ image pertaining to the ith reflection spectrum. Here, P assumes a value of 4, with odd indices representing Re($S_{ij}$) and even indices corresponding to Im($S_{ij}$). The variable Q denotes the 100 x 100 $S_{ij}$ images.

**Deep learning architecture.** The INN is divided into two modules, i.e., a CNN module for the inverse design and a forward module for the forward mapping. For the CNN module, there are 7 convolution layers and 3 deconvolution layers, in which the seventh layer are used for fusing the processed features and further exporting the metasurface design $\mathbf{R}$ which is binarized to comply with the physical design constraint. A sigmoid function is applied to the final layer of the forward network to ensure the output falls within the [0, 1] range. The layers are followed by batch normalization and rectified linear unit (ReLU) layers. The inclusion of batch normalization and ReLU layers expedites the training process and bolsters the network's robustness, effectively mitigating challenges associated with vanishing gradients and overfitting. All models are trained on a CPU of Intel (R) Core (TM) i7-8700K and a graphics processing unit (GPU) of NVIDIA GeForce RTX 2080 SUPER.

295
296 Acknowledgments

297 The work at Zhejiang University was sponsored by the Key Research and Development Program of
298 the Ministry of Science and Technology under Grants No. 2022YFA1404704, 2022YFA1405200, and
299 2022YFA1404902, the National Natural Science Foundation of China (NNSFC) under Grants No.
300 61975176, the Key Research and Development Program of Zhejiang Province under Grant
301 No.2022C01036, the Top-Notch Young Talents Program of China and the Fundamental Research
302 Funds for the Central Universities.

303 Conflict of interest

304 The authors declare no competing financial interests.

305 Contributions

306 C.Q., Y.J., and H.C. conceived the idea. Y.J. performed the simulation and machine learning part. Z.F.
307 provided assistance with the machine learning. Y.J. and C.Q. wrote the paper. P.d.H. contributed to
308 the discussion of the results and writing of the paper. C.Q. and H.C. supervised the project.

309 Data availability

310 The data that support the findings of this study are available from the authors on reasonable request.

311 References

312 1. LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. *Nature* **521**, 436–444 (2015).

313 2. Otter, D. W., Medina, J. R. & Kalita, J. K. A survey of the usages of deep learning for natural
314 language processing. *IEEE Trans. Neural Netw. Learn. Syst.* **32**, 604-624 (2021).

315 3. He, K. M. et al. Deep residual learning for image recognition. *IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR)*, 770-778 (2016).
316

317 4. Sutton, R. S. & Andrew, G. B. Reinforcement learning: An introduction. MIT press, (2018).

318 5. Gao, L. et al. A bidirectional deep neural network for accurate silicon color design. *Adv. Mater.* **31**, 1905467 (2019).
319

320 6. Ma, W. et al. Deep learning for the design of photonic structures. Nat. Photon. **15**, 77-90 (2021).

321 7. Li, L. L. et al. Machine-learning reprogrammable metasurface imager. *Nat. Commun.* **10**, 1082 (2019).
322

323 8. del Hougne, P. et al. Learned integrated sensing pipeline: reconfigurable metasurface transceivers as trainable physical layer in an artificial neural network. *Adv. Sci.* **7**, 1901913 (2020).
324

325 9. Qian, C. et al. Deep-learning-enabled self-adaptive microwave cloak without human intervention. *Nat. Photon.* **14**, 383-390 (2020).
326

327 10. Chen, C. et al. Neural network assisted high-spatial-resolution polarimetry with non-interleaved chiral metasurfaces. *Light Sci. Appl.* **12**, 288 (2023).
328

329 11. Khatib, O. et al. Deep learning the electromagnetic properties of metamaterials - A comprehensive review. *Adv. Funct. Mater.* **31**, 2101748 (2021).
330

331 12. Fan, Z. X. et al. Transfer-learning-assisted inverse metasurface design for 30% data saving. *Phys. Rev. Appl.* **18**, 024022 (2022).
332

333 13. Krasikov, S. et al. Intelligent metaphotonics empowered by machine learning. *Opto-electron. Adv.* **5**, 210147 (2022).
334

335 14. Fan, Z. X. et al. Homeostatic neuro-metasurfaces for dynamic wireless channel management. *Sci. Adv.* **8**, eabn7905 (2022).
336
337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375

376

377

378

15. Wang, Z. D. et al. Demonstration of spider-eyes-like intelligent antennas for dynamically perceiving incoming waves. *Adv. Intell. Syst.* **3**, 2100066 (2021).

16. Lin, C. H. et al. Metasurface-empowered snapshot hyperspectral imaging with convex/deep (CODE) small-data learning theory. *Nat Commun.* **14**, 6979 (2023).

17. Jiang, J. Q., Chen, M. K. & Fan, J. A. Deep neural networks for the evaluation and design of photonic devices. *Nat. Rev. Mater.* **6**, 679-700 (2021).

18. Zhang, J. et al. Heterogeneous transfer-learning-enabled diverse metasurface design. *Adv. Opt. Mater.* **10**, 2200748 (2022).

19. Yuan, X. Y. et al. Training large-scale optoelectronic neural networks with dual-neuron optical-artificial learning. *Nat. Commun.* **14**, 7110 (2023).

20. Qian, C. et al. Dynamic recognition and mirage using neuro-metamaterials. *Nat. Commun.* **13**, 2694 (2022).

21. Wu, N. X. et al. Pushing the limits of metasurface cloak using global inverse design. *Adv. Opt. Mater.* **11**, 2202130 (2023).

22. Ji, W. Y. et al. Recent advances in metasurface design and quantum optics applications with machine learning, physics-informed neural networks, and topology optimization methods. *Light Sci. Appl.* **12**, 169 (2023).

23. Chen, J. T. et al. Correlating metasurface spectra with a generation-elimination framework. *Nat. Commun.* **14**, 4872 (2023).

24. Nadell, C. C. et al. Deep learning for accelerated all-dielectric metasurface design. *Opt. Express* **27**, 27523-27535 (2019).

25. Liu, D. J. et al. Training deep neural networks for the inverse design of nanophotonic structures. *ACS Photonics* **5**, 1365-1369 (2018).

26. Tan, Q. Z. et al. Solving multivariable equations with tandem metamaterial kernels. *Prog. Electromagn. Res.* **175**, 139-147 (2022).

27. Zhu, X. Y. et al. Realization of index modulation with intelligent spatiotemporal metasurfaces. *Adv. Intell. Syst.* **5**, 2300065 (2023).

28. Jia, Y. T. et al. In situ customized illusion enabled by global metasurface reconstruction. *Adv. Funct. Mater.* **32**, 2109331 (2022).

29. Zhu, R. C. et al. Phase-to-pattern inverse design paradigm for fast realization of functional metasurfaces via transfer learning. *Nat. Commun.* **12**, 2974 (2021).

30. Qian, C. et al. Performing optical logic operations by a diffractive neural network. *Light Sci. Appl.* **9**, 59 (2020).

31. Tan, Q. Z., Qian, C. & Chen, H. S. Inverse-designed metamaterials for on-chip combinational optical logic circuit. *Prog. Electromagn. Res.* **176**, 55-56 (2023).

32. Lin, P. J. et al. Enabling intelligent metasurfaces for semi-known input. *Prog. Electromagn. Res.* **178**, 83-91 (2023).

33. Jiang, J. Q. & Fan, J. A. Global optimization of dielectric metasurfaces using a physics-driven neural network. *Nano Lett.* **19**, 5366-5372 (2019).

34. Qian C. & Chen, H. S. A Perspective on the next generation of invisibility cloaks—intelligent cloaks. *Appl. Phys. Lett.* **118**, 180501 (2021).

35. Tao, H. et al. Perfect anomalous reflectors at optical frequencies. *Sci. Adv.* **8**, 1-9 (2022).
379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395

396

397

398

36. Deng, Y. et al. Neural-adjoint method for the inverse design of all-dielectric metasurfaces. *Opt. Express* **29**, 7526-7534 (2021).

37. Wiecha, P. R. et al. Deep learning in nano-photonics: inverse design and beyond. *Photon. Res.* **9**, B182-B200 (2021).

38. Khatib, O. et al. Learning the physics of all-dielectric metamaterials with deep Lorentz neural networks. *Adv. Optical Mater.* **10**, 2200097 (2022).

39. Sol, J. et al. Experimentally realized physical-model-based wave control in metasurface-programmable complex media. *arXiv:2308.02349* (2023).

40. Cai, T. et al. Ultrawideband chromatic aberration-free meta-mirrors. *Adv. Photonics* **3**, 016001 (2021).

41. Deng, F. S. et al. Bessel beam generated by the zero-Index metalens. *Prog. Electromagn. Res.* **174**, 89-106, 2022

42. Fodor, J. A. & Pylyshyn, Z. W. Connectionism and cognitive architecture: A critical analysis. *Cognition* **28**, 3-71 (1988).

43. Jia, Y. T. et al. A knowledge-inherited learning for intelligent metasurface design and assembly. *Light Sci. Appl.* **12**, 82 (2023).

44. Ruder, S. An overview of gradient descent optimization algorithms. *arXiv preprint arXiv:1600.04747* (2016).

45. Kingma, D & Ba, J. Adam: a method for stochastic optimization. *arXiv preprint arXiv:1412.6980* (2014).
<figure><img src="image_3.png" /><figcaption>**Fig. 1 | Comparison of conventional and synthetical neural network.** Neural networks, emulating the biological nervous system, consist of interconnected neurons that process input through weighted mechanisms to generate the output. In conventional neural networks, the network is typically indivisible and necessitates an abundance of data for training. Our synthetical neural network shifts this paradigm by breaking down the solution space. Each subproblem is assigned to a specific neural network (e.g., NN 1~NN 4). These neural networks can be freely assembled to adapt to different tasks. When faced with a completely new task, selecting appropriate NNs and assembling them in a specific order allows for the swift synthesis of an entirely new network, as illustrated by the free assembly of NN 1/2/4 and the newly added NN 5.</figcaption></figure>
<figure><img src="image_4.png" /><figcaption>**Fig. 2 |** Design and architecture of the synthetical neural network. **a,** In metasurface inverse design, the conventional methodology involves utilizing the reflection spectrum ($S_{11}$ parameter) as the input and obtaining the metallic resonant surface as the output through machine learning-based optimization. In the conventional method, the task-specific neural network (TNN 1) must be discarded when transitioning to a new task (Target 2) with a new required TNN 2, presenting a time-consuming challenge. **b,** Our method treats metasurfaces as composite structures assembled from different "parent" segments. Each segment has an associated inverse design network (INN), systematically cataloged in a "gene" library. When confronted with a new task (Target 2), we assemble the pre-trained INN1 (old) and the INN2 (new) corresponding to $R_{10}^8$ using the ASNN. This assembly method mitigates data requirements to a certain extent, and each newly added INN can be stored in the gene library for future reuse. **c,** The INN and ASNN are structured as CNNs with an encoder-decoder configuration, where the input/output consists of compressed $S_{11}$ images (including real and imaginary parts) as detailed in Fig. 3. The forward design networks included in the INN addresses the non-uniqueness challenge of inverse design.</figcaption></figure>
<figure><img src="image_5.png" /><figcaption>b Training result of SNN with discretized $S_{11}$</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>c</figcaption></figure>

<figure><img src="image_7.png" /></figure>

425

426

427

428

429

430

431

432

433

434

<caption>**Fig. 3** | Pre-processing of spectral data. **a**, Two distinct processing methods for spectral data. **Method 1** is a simple 1D discretized representation of the continuous physical spectrum. **Method 2** formats the representation from **Method 1** into a 100 x 100 image. **b**, Results of ASNN training with **Method 1**. The network only achieves an accuracy of 55.9%. Examining **Test sample 1** to observe the training effect, the notable discrepancy between predicted (dotted line) and actual values (solid line) further illustrates the suboptimal outcome of the training. **c**, Illustration of the CNN-based encoder. **Method 2** is more suitable for CNN-based image processing. Through convolution/deconvolution operations on local pixel information, the network gradually aggregates the knowledge of sharp resonant features at higher layers.</caption>
<figure><img src="image_8.png" /></figure>

<figure><img src="image_9.png" /></figure>

<figure><img src="image_10.png" /></figure>

<figure><img src="image_11.png" /><figcaption>**Fig. 4** | Training results for INN/ASNN. a, Training results for INN 1 over training epochs, reaching an accuracy of 99.4%. The accuracy for INN 2 is 99.7%. b, Predicted real/imaginary $S_{11}$ images of four samples by INN 1/2, with embedded insets depicting the corresponding ground-truth images. The close correspondence between the ground-truth images and the predicted images highlights the high accuracy. The corresponding $R_8/R_{10}^8$ patterns of the predicted metasurface are displayed below. c, Training results for the ASNN over the training epochs, reaching an accuracy of 96.7%. d, Predicted real (blue)/imaginary (yellow) $S_{11}$ images of $R_{10}$ by the synthetic neural network, with embedded insets depicting the ground truth image and the accuracy. The predicted $R_{10}$ patterns are displayed below. The corresponding split $S_{11}$ images of the two "parent" components ($R_8/R_{10}^8$) are connected by arrows.</figcaption></figure>
<figure><img src="image_12.png" /><figcaption>**Fig. 5 | Scalability of synthetical neural network.** a, For the new $R_{14}$ design task, there is no need to collect and train new data and networks specific to $R_{14}$. The process involves reusing two existing INNs, namely INN 1 and INN 2, based on the framework depicted in Fig. 2a. However, the introduction of a new INN 3, dedicated to the $R_{14}^{10}$ annular ring, is necessary. Subsequently, a new ASNN' to assemble all three INNs is trained to accomplish the inverse design task for $R_{14}$. b, Training results for INN 3 over the training epochs, reaching an accuracy of 99.7%. The blue and red lines represent the training and validation losses, respectively. c, Predicted real/imaginary $S_{11}$ images of three samples with INN 3 for the new $R_{14}^{10}$ "parent" component, with embedded insets depicting the ground-truth image; the achieved accuracies are stated. The predicted $R_{14}^{10}$ "parent" components are displayed below. d, Training results for the new ASNN' over the training epochs, reaching an accuracy of 95.5%. e, Predicted real $S_{11}$ images of one sample by the new synthetical neural network, with embedded insets depicting the ground-truth image. On the right side, the corresponding predicted "parent" component, $R_{14}$, is displayed. Arrows connect the corresponding split $S_{11}$ images of $R_8/R_{10}^8/R_{14}^{10}$. f, Training results for the conventional neural network over the training epochs. It is in a serious underfitting state, and its accuracy is only 70.5%. g, Predicted real/ imaginary $S_{11}$ images by conventional network.</figcaption></figure>