## 2 Histogram Based Clustering  
Figure 1 shows the flow diagram of the proposed HBC algorithm. For the sake of illustration, we employed 10,000 16-QAM symbols that were obtained by applying an amplitude dependent nonlinear phase rotation (\(\Delta\phi[n]=3A^{2}[n]\), where \(A[n]\) is the amplitude of the symbol) and including complex additive noise with a signal-to-noise ratio (SNR) of 20 dB. It is worth noting that this simple model was employed only for demonstration purposes because by neglecting the intersymbol interference, the model did not accurately represent the system. In the next section, we will describe the model used to consider simultaneously nonlinear and linear effects that result in more complex symbol distributions. The resultant constellation is shown in Figure 1a. The histogram of the unlabelled symbols was calculated, in this case, using 40 bins in the in-phase and quadrature directions,  
resulting in the contour plot presented in Figure 1b, where the 16 clusters can be clearly identified. Once the histogram was calculated, the lowest value contour line led to the desired number of clusters, 16 in the case of 16-QAM. The value of this contour line represents the optimal point density. A lower point density does not allow the correct recovery of all the clusters, while a larger value results in a too conservative criterion that leads to poorer performance, as the decision regions are not well matched to the data. This optimal density searching mechanism is, indeed, one of the main advantages over other density based clustering algorithms, such as DBSCAN. In fact, DBSCAN is intended for data clustering where the number of clusters is a priori unknown and the minimum density of clusters is fixed through the values of \(\epsilon\) and \(k_{min}\). This was not our case, where the number of clusters was known and the minimum density was dependent on the distortion and, therefore, also on the launch power level. The determined cluster borders are superimposed on the histogram in Figure 1c, showing that, especially the constellation points with stronger distortion, those in the periphery, the boundaries of the different clusters were closer to each other. After finding the cluster boundaries, each cluster was identified by the points that formed its boundary instead of a centroid as in k-means. Figure 1d shows the boundaries of the different clusters on top of the received constellation, where it can be clearly seen that the obtained cluster boundaries encompassed most of the points of their respective clusters. Once the boundary points for each cluster were found, the distance from each received symbol to them was calculated, and the class of the boundary point with the shortest distance was assigned to the symbol. The classified constellation is shown in Figure 1e.  
In a systematic way, the proposed HBC algorithm consisted of the following four steps:  
1. Calculation of the 2D histogram of the in-phase and quadrature components of the \(N_{S}\) received distorted symbols.
2. Find the lowest contour line in the histogram that results in \(M\) isolated islands, \(M\) being the number of clusters to be identified.
3. Assign a class ID to the values of the boundary for each island.
4. For each received symbol, find the closest boundary point and associate it with its class ID.  
Figure 1: Flow diagram of the proposed clustering algorithm. (**a**) Distorted input 16-QAM constellation. (**b**) Calculated 2D histogram. (**c**) Optimum boundaries superimposed on the 2D histogram. (**d**) Boundaries for each cluster on top of the received distorted constellation. (**e**) Classified constellation.