Whenever a theory simply bestows structure on the mind a priori it is making an extravagant claim. However, it is not at all clear that nonlearnability has the status of a deductive proof just yet. They are not unlearnable in principle. Well here the problem may be deeper, and I, at least, think it resides in part in the miscasting of symbolism and connectionism as rivals for cognitive hegemony, rather than potential collaborators. Let us quickly set aside their trivial attempts to subsume one another.
Symbolists say nets are just an alternative hardware for doing computation.
That may be one possibility, but it is certainly not the one that connectionists are exploiting when they use nets to learn. Connectionists say nets have the power of Turing machines. Perhaps, but not when they are used as nets, rather than as a hardware for implementing a Turing machine. Symbolists say most nets are just symbolically simulated nets anyway. True, but they could be implemented as real nets too.