This paper shows that the distinction between inputs and outputs is not this simple.
Some features are more useful as extra outputs than as inputs.
By using a feature as an output we get more than just the case values but can learn a mapping from the other inputs to that feature. For many features this mapping may be more useful than the feature value itself. This paper shows the same effect on a real problem, and presents a means of determining what features can be used as extra outputs. A Note on Learning Vector Quantization - de Sa, al.