## 2 Multi-Attribute Recognition Using Exclusive and Correlated Relationships
### Multi-Task Learning for Independent Attributes  
As the next step, in order to learn multiple tasks at the same time, the conventional cross-entropy error function for single task is extended to multiple tasks. Figure 2 illustrates the network for training two tasks at the same time. Let us assume that we have \(T\) classification tasks \(\mathbf{T_{f}}(t=1\ \ldots\ T)\), and each task \(\mathbf{T_{f}}\) is composed of \(M_{t}\) mutually exclusive attributes \(A_{tm}(t=1,\ldots,T,\ m=1,\ldots,M_{t})\). We assign one output node for each attribute so that the whole network for \(T\) tasks has \(\sum_{t=1}^{T}M_{t}\) output nodes. The target value of \(m\)th output node for \(t\)th task can be denoted as \(y_{tm}^{n}\) and satisfies the conditions:  
\[\sum_{m=1}^{M_{t}}y_{tm}^{n}=1\ (t=1,\ \ldots,T)\,y_{tm}^{n}\in\{0,1\}. \tag{3}\]  
In order to satisfy these conditions, the output value of the node corresponding to attribute \(A_{tm}\) is defined by using a task-wise softmax activation function such as  
\[f_{tm}(\mathbf{x_{n}})=\frac{e^{\text{i}tm}}{\sum_{i=1}^{M_{t}}e^{\text{i}u_{i}}}\, \tag{4}\]  
Figure 1: A network structure with single task \(\mathbf{T}\) for recognizing \(\mathbf{M}\) mutually exclusive attributes.  
where \(u_{tm}^{n}\) is the weighted sum of input injected to the output node for the attribute \(A_{tm}\) when an input \(\mathbf{x_{n}}\) is given. Since the softmax function is applied not to all the output nodes but to the task-wise group, we also need to modify the conventional cross-entropy error function so that summation is applied task-wisely, which can be written as  
\[E_{mce}(\mathbf{\theta}) = -\sum_{n=1}^{N}\sum_{t=1}^{T}\sum_{m=1}^{M_{t}}y_{tm}^{n}\ln f_{tm }(\mathbf{x_{n}},\mathbf{\theta})\, \tag{5}\]  
where \(T\) is the number of tasks and \(M_{t}\) is the number of attributes in \(t\)th task.  
This extension to multi-task learning from single task learning for a group of exclusive attributes is limited in the sense that it only utilizes the mutual relationship between attributes, and does not consider the relationship between tasks. The simple summation of the task-wise cross-entropy function defined by Equation (5) is derived by the assumption that the tasks are not correlated and the target random vectors \(\mathbf{y}_{t}=[y_{t1}\dots y_{M_{t}}]\) (\(t=1\dots T\)) are mutually independent to each other. However, this strong assumption does not seem plausible in real world applications. For example, it can be easily assumed that the two random vectors of two tasks for recognizing the attributes of eye color and the attributes of hair colors are closely related note that a person with blond hair often has blue eyes. Moreover, these two recognition tasks are also associated with race recognition tasks in that Caucasians are likely to have blond hair with blue eyes. As mentioned in [7], the mutual relationship between tasks can make some influence on the performance in each task, we can expect to get better recognition performance by multi-task learning considering the relationship between tasks. Nevertheless, it is hard to define the relation in the learning model since this relationship is quite data-specific. In the next section, we propose an approach to overcome this difficulty.