# Machine Learning Control of Modulation Instability in Optical Fiber

Yassin Boussafa, Lynn Sader, V. T. Hoang, Bruno P Chaves, Alexis Bougaud, R. Haldar, M. Fabert, A. Tonello, J.M. Dudley, M. Kues, et al.

► To cite this version:

Yassin Boussafa, Lynn Sader, V. T. Hoang, Bruno P Chaves, Alexis Bougaud, et al. Machine Learning Control of Modulation Instability in Optical Fiber. 2025 Conference on Lasers and Electro-Optics Europe & European Quantum Electronics Conference (CLEO/Europe-EQEC), Jun 2025, Munich, Germany. 10.1109/CLEO/Europe-EQEC65582.2025.11110883. hal-05240453

HAL Id: hal-05240453

https://hal.science/hal-05240453v1

Submitted on 11 Sep 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Machine Learning Control of Modulation Instability in Optical Fiber

Y. Boussafa¹, L. Sader¹, V. T. Hoang¹, B. P. Chaves¹, A. Bougaud¹, R. Haldar²,³, M. Fabert¹, A. Tonello¹,
J. M. Dudley⁴, M. Kues², B. Wetzel¹

1. XLIM Research Institute, CNRS UMR 7252, Université de Limoges, Limoges 87060, France

2. Institute of Photonics, Leibniz University Hannover, Hannover, Germany

3. School of Electrical and Computational Sciences (SECs), Indian Institute of Technology Bhubaneswar (IITBBS), Odisha 752050, India

4. Institut FEMTO-ST, CNRS, Université Marie et Louis Pasteur, Besançon, France

The fields of ultrafast and nonlinear photonics have recently experienced remarkable progress, with machine learning emerging as a key driver in many research efforts. These advances particularly benefit from the development of efficient characterization techniques combined with advanced optical pulse shaping methods designed to control the nonlinear dynamics involved in guided pulse propagation. A notable example is modulation instability (MI), a well-known nonlinear process of guided optics [1]. Based on four-wave mixing, MI is characterized by the emergence of new spectral components, originating from coherent signals (induced MI) or appearing spontaneously through noise amplification (spontaneous MI). In realistic conditions, MI intrinsically involves incoherent and complex processes, ultimately making its experimental control difficult. Here, we present an overview of our recent results focusing on machine learning strategies for the characterization, control, and optimization of incoherent signals generated via modulation instability in optical fibers. Specifically, we numerically and experimentally study the control of spontaneous MI by optical seeding, using weak coherent signals whose parameters are adjusted by machine-learning optimization techniques.

<figure><img src="image_2.png" /><figcaption>**Fig. 1** (a) Experimental setup for ML-based optimization of MI via coherent optical seeding and DFT characterization. (b,c) GA-based optimization of correlation between two wavelengths in the output MI spectra (i.e. white dots in the experimental Pearson correlation map shown in (c)). (d) ANN inference of coherent seed properties: example of the ANN prediction of one input seed wavelength from the output spectral correlation maps. The ANN was trained from experimentally measured (incoherent) DFT spectral traces.</figcaption></figure>

In our optical setup, reconfigurable coherent optical seeding is performed via a programmable spectral pulse-shaper (Finisar - Waveshaper 4000A) used to filter a broadband spectrum and tailor a picosecond pump pulse [2]. Mutually coherent seed pulses are further added to this primary pump pulse, each with tunable central wavelengths and spectral phases. After propagation in a highly nonlinear fiber, the incoherent MI-broadened spectrum can be characterized via dispersive Fourier transform (DFT), enabling shot-to-shot spectral fluctuations to be measured in real-time via ultrafast optoelectronic detection [2, 3]. After suitable processing, we show that controlled coherent optical seeding can be adjusted via a variety of machine-learning approaches to tailor and optimize incoherent spectral broadening dynamics. In particular, genetic algorithms (GA) can be leveraged to iteratively optimize specific correlation features between the output spectral fluctuations. Conversely, we demonstrate that artificial neural networks (ANN) can be trained from these incoherent dynamics, to predict the output spectral correlation features of noise-driven MI dynamics from different seeding scenarios (without requiring further real-time spectral monitoring) or to infer elusive input seed properties (competing with noise and concealed during fiber propagation) [4]. Besides enabling practical control of ubiquitous incoherent structures, our results provide a new toolset for the exploration of complex optical phenomena while easing the requirements of (extensive) time-consuming simulations or advanced real-time measurements, with a potential fundamental impact across a broader range of nonlinear systems and noise-driven architectures.

## References

[1] J. M. Dudley, F. Dias, M. Erkintalo, and G. Genty "Instabilities, breathers and rogue waves in optics," Nat. Photonics 8, 755(2014).

[2] L. Sader, et al. "Single-Photon Level Dispersive Fourier Transform: Ultrasensitive Characterization of Noise-Driven Nonlinear Dynamics," ACS Photonics 10, 3915 (2023).

[3] T. Godin, et al. "Recent advances on time-stretch dispersive Fourier transform and its applications," Adv. Phys. X 7, 2067487 (2022).

[4] Y. Boussafa, et al. "Deep learning prediction of noise-driven nonlinear instabilities in fibre optics," Submitted (2024)

This work received funding from his work has received funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme under grant agreement 950618 (STREAMLINE) and 947603 (QFreC). B.W. acknowledges the support of the French ANR through the OPTIMAL project (ANR-20-CE30-0004) and the Région Nouvelle Aquitaine (SPINAL project). This research benefited from the support of the Platinom platform, with funding the European Union and the Nouvelle Aquitaine council under the PILIM program.