# Short Words for Writer Identification Using Neural Networks
## 6 Discussion of Space Separability  
The capabilities of the conventional multilayer neural networks were described in an early tutorial paper by [PERSON] [24]. In that work, it was clearly stated that:  
1. The first layer of perceptrons draws in the N-D feature space linear subspaces of (N\(-\)1) D dimensionality. 2. The second (and the third, if necessary) layers of perceptrons combine the above linear (N\(-\)1) D subspaces to create any type of convex and non-convex decision regions in the original feature space. When the cluster populations are well separated, such as those shown in Figure 7a,b, it is expected that the neural structure that will be suitable for separating the feature space will be the simplest possible. For example, if two populations are to be separated, only one linear subspace is needed, as shown in Figure 8a, while with two subspaces in a general position, we can divide the space into four regions (Figure 8b), and with three subspaces in a general position, we can divide the 2-D feature space into seven regions (Figure 8c). \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \(\lambda\): & 363 & 121 & 47 & 32 & 14 & 11 & 7 & 5 & 4 & 1 \\ \hline \(\sqrt{\lambda}\): & 19 & 11 & 6.8 & 5.6 & 3.7 & 3.3 & 2.6 & 2.2 & 2 & 1 \\ \hline \end{tabular}
\end{table} 
Table 1: Ten largest eigenvalues corresponding to the most prominent directions for cluster separability in the 50-D feature space, for the word “the”. Figure 7: Writer discrimination results. The clusters correspond to the same word “the”, written 6 times by 10 different writers. The FLD is used to project the clusters onto three dimensions with the maximum separability. The measurements were obtained using morphological structuring element of length 3 and 5; consequently, dimensionality reduction from 50 to 3 dimensions was performed. Two different aspects of the same 3-D feature space are given in (**a,b**). According to the previous material, the question which arises is:  
\"How many separable regions can be obtained in a n-dimensional feature space when m hyperplanes of dimensions n\(-\)1 are employed is a general position?\"  
The answer is obtained from the following proposition [25]:  
_Let \(A\) be an n-dimensional arrangement of m hyperplanes with n-1 dimensions in a general position. The number r(A) of well-separated regions determined in the n-dimensional space for the general case is_  
\[r(A)=1+m+\binom{m}{2}+\cdots\binom{m}{n} \tag{6}\]  
For example, if \(n=50\), as in the case of features extracted in Sections 4 and 5, then for \(m=3\) subspaces, we obtain a number of well separable regions as  
\[r(A)=1+\binom{3}{1}+\binom{3}{2}+\binom{3}{3}=8 \tag{7}\]  
while for m = 4 subspaces, the number r(A) of well separated regions is  
\[r(A)=1+\binom{4}{1}+\binom{4}{2}+\binom{4}{3}+\binom{4}{4}=1+4+6+4+1=16 \tag{8}\]  
According to Equations (7) and (8), the number of (n\(-\)1)-D subspaces required to separate the ten populations shown in Figure 7a,b should be more than three to obtain ten well-separated regions. However, this inference is not valid since the neural structure combines the (n\(-\)1)-D linear subspaces drawn with the first layer of perceptrons in order to form more complicated separable regions. This is depicted in the following using the populations of the writers shown in Figure 7a,b, on the 2-D space, on which the clusters have their best separability according to the FLD (5). Figure 9 shows the position of the clusters in the two highest separability dimensions, while Figure 10 depicts the way these populations have been separated using a neural structure of two layers with three and four perceptrons, respectively. Obviously, the linear subspaces that separate the populations are curved according to the cluster's shape. As it is depicted in Figure 10, a very simple neural structure can separate the ten clusters with well-formed decision regions.