###### Abstract  
The future evolution of wireless networks, throughout the 5G era and beyond, will require the expansion and augmentation of millimetre-wave systems for both terrestrial and satellite communications. Photonic technologies offer a cost efficient and high bandwidth platform for millimetre-wave carrier generation and distribution, but can introduce high levels of phase noise through optical heterodyning, which is highly problematic for mobile signal waveforms. In this work, a detailed analytical model of a hybrid photonic/mm-wave system is developed and discussed. Through careful system design, the system is found to support both 5G compatible multi-carrier (OFDM) and single carrier (APSK) modulation at 60 GHz. APSK is found to offer higher tolerance mm-wave phase noise compared to OFDM, ultimately easing optical linewidth restrictions to \(\sim\)30 kHz. The model is extended to include a novel millimetre wave phase noise cancelling receiver, which is shown to significantly alleviate these restrictions even further--enabling phase noise free mm-wave operation for optical linewidths up to \(\sim\)2 MHz. Detailed analysis and discussion of this extended system lead to the establishment of a theoretical relationship between the mm-wave receiver design and the achievable system performance in terms of error vector magnitude (EVM). Excellent matching of the predicted theoretical with simulated performances is shown. millientre wave; optical heterodyne; beyond 5G; radio-over-fiber; satellite communications 00000-0000-0000-0000-0000-0000-0000-00000-00000-00000-00000-00000-00000-00000-00000-00000-00000-00000-00000-00000-00000-00000-000000-00000  
but in order for these links to complement future terrestrial mobile service delivery, further expansion of the V-band (40-75 GHz) and the W-band (75-110 GHz) will likely be required to deal with predicted demand [3]. The extent to which mm-wave communications can be leveraged for future wireless systems depends greatly on the ability to easily generate mm-wave frequency carriers. The difficulty and expense currently associated with generating high quality carriers in this frequency range, by purely electronic means, hinder the potential wide deployment of these systems. An alternative method for mm-wave generation is optical heterodyning, which involves the photo-mixing of two optical carriers, spaced by a mm-wave frequency, on a high speed photo diode (PD)--producing a beat term mm-wave carrier at the output of the PD. The relative ease of optical carrier generation, operation over a wide bandwidth/frequency range, inherent compatibility with optical distribution technologies, avoidance of costly high frequency electronic components, and the continued development of small form factor integrated microwave/mm-wave photonic components make heterodyning a highly promising contender for mm-wave systems, in the 5G era and beyond [4; 5]. Optical heterodyning has long been proposed as a method for mm-wave and terahertz carrier generation [6; 7], with many demonstrations showcasing its ability to form the basis for high bandwidth, high throughput wireless systems [8]. However, studies of the compatibility of such photonic systems with the modulation formats required for future mobile service delivery have been limited to date. Our own recent works have shown how mm-wave phase noise derived from the optical domain, through the heterodyne process, can severely degrade multi-carrier modulation (MCM) mobile formats [5; 9; 10]. In an effort to mitigate this system limitation, we recently proposed and experimentally demonstrated an analog noise-cancelling photonic/mm-wave system design, which enabled phase noise free reception of an MCM mobile signal at 61 GHz [11]. In this work, detailed analytical models of a hybrid photonic/mm-wave system, employing both a traditional stand-alone local oscillator (LO) based receiver, and the phase noise cancelling (PNC) receiver described in [11], are presented.