# Hierarchical Concept Learning by Fuzzy Semantic Cells
## 3 Method
### Optimization of Fuzzy Semantic Cells  
In this section, we will introduce the principles of learning FSCs. The proper partition \(LA\) means this partition makes the concepts of elements belonging to each \(L_{i}\) are as consistent as possible. That is to say, the concept purity of elements covered by every \(L_{i}\) is as high as possible. Figure 2: The hierarchical structure diagram of concept learning model. To this end, for each concept (category), we compute the sum of the membership of all the elements under that concept to every fuzzy semantic cell \(L_{i}\) marked \(r_{ij}\). In fact, \(r_{ij}\) approximates the average number of samples covered by \(L_{i}\) in \(D_{j}\):  
\[r_{ij}=|L_{i}\cap D_{j}|=\sum_{x\in D_{j}}\mu_{L_{i}}(x)\]  
Then we normalize all \(r_{ij}\)'s for every \(L_{i}\) to estimate the probability that samples covered by \(L_{i}\) in \(D_{j}\):  
\[p_{ij}=\frac{r_{ij}}{\sum_{j=1}^{K}r_{ij}}\]  
As a result, the relative entropy of every \(L_{i}\) is defined as:  
\[\bar{H}(S\mid L_{i}\cap DB)=-\sum_{j=1}^{K}p_{ij}\ln p_{ij}\]  
We minimize \(\bar{H}(S\mid L_{i}\cap DB)\) to get the prior distribution for every fuzzy semantic cell. As mentioned above, our goal is to get the \(L_{i}\) which maximizes the concept purity. In other words, we need to find a partition of fuzzy semantic cells that is highly consistent with the concept partition. Our solution is inspired by the work [16]. In [16], the authors give the following definition of consistency of dataset segmentation:  
Definition 4: Let \(X\) be a finite dataset and let \(X_{l}\subset X\) be the set of labeled data points that is partitioned into \(Z=\{Z_{1},\dots,Z_{m}\}\). A partition \(Y=\{Y_{1},\dots,Y_{k}\}\) of \(X\) is consistent with \(Z\), if for every \(Y_{i}\) there exists at most one \(Z_{j}\) such that \(Z_{j}\cap Y_{i}\
eq\emptyset\). Based on this, they proved that conditional entropy \(H(Z\mid Y,X_{l})\) can be used to measure the consistency of dataset segmentation:  
\[H(Z\mid Y,X_{l}) =\sum_{i=1}^{k}\frac{|Y_{i}\cap X_{l}|}{|X_{l}|}H(Z\mid Y_{i}\cap X _{l})\] \[=\sum_{i=1}^{k}\frac{|Y_{i}\cap X_{l}|}{|X_{l}|}\sum_{j=1}^{m} \frac{|Y_{i}\cap Z_{j}|}{|Y_{i}\cap X_{l}|}(-\log\frac{|Y_{i}\cap Z_{j}|}{|Y_{ i}\cap X_{l}|})\]  
The smaller the conditional entropy is, the higher the consistency is. Inspired by this idea, we give the following definition of partition consistency of \(LA\) and \(S\):  
Definition 5: Let \(DB\subset\Omega\) be a finite dataset and \(DB\) is partitioned into \(S=\{D_{1},\dots,D_{K}\}\). A partition \(LA=\{L_{1},\dots,L_{M}\}\) of \(\Omega\) is consistent with \(S\), if for every \(L_{i}\) there exists at most one \(D_{j}\) such that \(D_{j}\cap L_{i}>0\). Then we introduce the conditional entropy to our objective function to ensure \(LA\) is consistent with \(S\). The conditional entropy in our scenario is defined as:  
\[\bar{H}(S\mid LA,DB) =\sum_{i=1}^{M}\frac{|L_{i}\cap DB|}{\sum_{k=1}^{M}|L_{k}\cap DB|} \bar{H}(S\mid L_{i}\cap DB) \tag{4}\] \[=-\sum_{i=1}^{M}\sum_{j=1}^{K}\frac{\sum\limits_{x\in D_{j}}\mu_{ L_{i}}(x)}{\sum\limits_{k=1}^{M}\sum\limits_{x\in DB}\mu_{L_{k}}(x)}\ln\frac{\sum \limits_{x\in D_{j}}\mu_{L_{i}}(x)}{\sum\limits_{x\in DB}\mu_{L_{i}}(x)}\]  
By minimizing the conditional entropy, we update the fuzzy semantic cells. The updating rule allows having the highest possible purity. At the same time, the parameter \(M\) controls the fine degree of FSCs. A proper number of \(M\) ensures that the fuzzy semantic cells are neither too rough nor too fine. \(LA\) is too fine means \(M\gg K\). In this case, too many FSCs correspond to the same category, although each \(L_{i}\) has high internal purity, and each fuzzy semantic cell covers a very small radius. This will cause great redundancy. Especially in extreme circumstances, the partition will be perfect when \(LA\) is separated into a small portion which only contains one element, that is to say, \(M\) is equal to the number of items \(x\).