_applied sciences_  
_Article_  
**Capacity Enhancement Analysis of an OAM-OFDM-SMM Multiplexed Free Space Communication System**  
**in Atmospheric Turbulence**  
**[PERSON] 1, [PERSON] 1,* [PERSON] 2, [PERSON] 3, [PERSON] 2,  and [PERSON] 2,* (c)**  
1 University School of Information, Communication & Technology, Guru Gobind Singh Indraprastha University, Sector 16 C, Dwarka, New Delhi 110078, India  
2 Department of Electrical Engineering, College of Engineering, Jouf University, Sakaka 72388, Saudi Arabia  
3 Department of Electronics and Communication Engineering, University Institute of Engineering, Chandigar University, Mohali 140413, India  
* Correspondence: [EMAIL_ADDRESS] (C.K.); [EMAIL_ADDRESS] (K.A.)  
**Clication:** [PERSON]; [PERSON], [PERSON]; [PERSON], [PERSON]; [PERSON], [PERSON]; [PERSON], [PERSON]; [PERSON], [PERSON] Capacity Enhancement Analysis of an OAM-OFDM-SMM Multiplexed Free Space Communication System in Atmospheric Turbulence. _Appl. Sci._**2023**, _13_, 3897. [[https://doi.org/10.3909/app13063897](https://doi.org/10.3909/app13063897)]([https://doi.org/10.3909/app13063897](https://doi.org/10.3909/app13063897))  
Academic Editors [PERSON] and [PERSON] Received: 24 December 2022  
Revised: 12 March 2023  
Accepted: 16 March 2023  
Published: 19 March 2023  
**Copyright**: © 2023 by the authors. Liconse MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license ([[https://creativecommons.org/licenses/py/4.0/](https://creativecommons.org/licenses/py/4.0/)]([https://creativecommons.org/licenses/py/4.0/](https://creativecommons.org/licenses/py/4.0/))).  
**Abstract:** To overcome atmospheric turbulence (AT) distortion during signal propagation through the optical link, orbital angular momentum (OAM) mode states employing multiple inputs and multiple outputs (MIMO) techniques have recently gained prominence in free space optical communication (FSO). As the various OAM modes propagate through the free space optical link, signal attenuation and crosstalk may occur, reducing system capacity and increasing the likelihood of bit errors. In this work, our objective is to propose a spectrally efficient, high-speed and channel capacity efficient crosstalk FSO communication system by combining the features of orthogonal frequency division multiplexing (OFDM), spatial mode multiplexing (SMM), and a mode diversity scheme into an existing OAM-FSO communication system. The incorporation of the OFDM-MIMO concept and spatial mode diversity into the existing OAM-MIMO-FSO system is extremely beneficial in enhancing the transmission capacity, mitigating multipath fading and atmospheric turbulence distortions. The Gamma-Gamma (GG) model is used to assess the performance of the proposed system under various atmospheric turbulence conditions in terms of the performance metrics such as BER vs. number of OAM states for different refractive index structure and Rytov constants, link distance, and an optical signal to noise ratio (OSNR). A FEC limit of 3.8 \(\times\) 10\({}^{-3}\) and a maximum link distance of 2 km are set to evaluate these performance parameters. Finally, the transmission capacity of the proposed system is compared to that of the existing MIMO and OAM-SMM-MIMO systems for different OSNR values under atmospheric turbulence conditions for the OAM state of \(l=+1\), yielding an overall improvement of 3.3 bits/s/Hz compared to conventional MIMO systems and 1.6 bits/s/Hz for the OAM-SMM-MIMO system.  
**Keywords:** adaptive MIMO equalization; bit error rate (BER); orbital angular momentum multiplexing; optical signal to noise ratio (OSNR); orthogonal frequency division multiplexing (OFDM); quadrature amplitude modulation (QAM); free space optical communications (FSO); spatial mode multiplexing (SMM); spatial mode diversity (SMD)