## 1 Introduction  
The spectral response of the periodic antenna structure placed in a dielectric homogeneous medium depends on the antenna geometry, the parameters of the medium, the angle of incidence, polarization, and the geometry of the excitation field. However, in many applications, the above number of parameters determining the operation of the antenna is not sufficient to be able to model its desired directional characteristics. Increasing the number of antenna structure parameters can be achieved by introducing a multilayer dielectric medium with a certain number of metallized periodic surfaces located on flat boundaries between the dielectric layers [1, 2]. The analysis of such a multilayer planar antenna system is the subject of this work. There are two complementary approaches to the analysis of such structures. In the first, the composite antenna system is analysed by constructing supermodes of the entire structure. In the second, the system is considered as a cascade assembly of flat discrete elements, i.e., the boundaries between two dielectrics, periodic metallized planes, and dielectric layers [3]. The latter approach leads to the definition of the scattering, transmission, or impedance matrix of the entire structure by cascading the corresponding matrices associated with the individual discrete elements of the antenna structure. It is particularly useful in modelling dielectric multilayer antenna walls, where the stored data on one planar antenna element can be used many times in the analysis of various antenna systems with modified parameters of other discrete structure elements. In practice, antenna arrays most often consist of periodic metal structures placed on or immersed in a dielectric multilayer medium. Changes in the size and shape of individual antenna elements enable efficient modeling of its spectral parameters. These changes can be additionally modeled by selecting the geometrical and physical parameters of the dielectric layered substrate and coating. In the case of large transverse dimensions, the antenna is in the order of 100 wavelengths and several dozen or more individual metal radiation elements. The radiation properties of the antenna approximately correspond to those of the unlimited antenna structure. Under the condition of periodicity of such an antenna and its excitation with linear, homogeneous phase modulation, the problem of scattering on such a structure comes down to the analysis of one basic antenna cell, usually in a spectral space spanned by a complete array of corresponding Floquet harmonics. For structures of smaller sizes and fewer elements, the effects related to their limitations, especially the edge effects occurring on their outer cells, may affect the antenna performance characteristics and should be considered in their modeling. So far, there is no known exact effective method for solving the scattering problem on such periodic constrained structures. Attempts to solve this problem by directly applying the spectral method of moments are only formally strict. This is because their accuracy is limited by the constraints imposed on the size of the matrices used in the procedures for solving them. Nevertheless, for antenna walls composed of a few or a dozen regular-shaped basic cells, this method gives good results. Other known methods of solving the scattering problem on larger periodic structures, such as modifications of the above-mentioned direct method by limiting the number of base functions and appropriate selection of their waveforms, or the iterative method imposed on the solution by successive application of the Fast Fourier transform, are approximate, slow convergent, and can only be used in some simple, easy-to-analyze cases. Currently, a computer program is being developed that will allow verification of the adopted assumptions. The generalized scattering matrix method was originally used in the analysis of waveguide problems, considering both propagating and vanishing [4].