>> Why do you believe there is infinite cat behaviour? Why would they evolve
that?

An infinity of behaviours is not a distinct ability that has evolved to
fulfill some purpose. Rather, it's the result of the animal interacting with
its environment. The number of possible such interactions is infinite - or,
well, most likely infinite.

An infinite number of combinations can arise from very simple processes- for
example, an automaton that generates strings in the aⁿbⁿ grammar (n a's
followed by n b's) can go on for ever. There is no reason to assume that a
complex mind in a complex environment will ever run out of combinations of
mind-states and world-states. Accordingly, there is no reason to believe we
will ever be able to collect examples of all those combinations, and represent
them in computer memory.

Edit: I'm not talking about a cat being elected governor here. Just ordinary
real-world behaviours, like all the ways a cat may chase a mouse, say. Try to
observe a cat and systematise its behaviour and see how welll you can do. Then
try to do it with a computer.

With a computer, it should be easier, right?

* * * *

I wouldn't say that the point of learning is to reduce infinity to something
manageable. I think it's more like animal minds, like ours, have some kind of
ability to pick out what is relevant to a learning task from the infinity of
available experiences (incidentally, that's the subject of my PhD thesis :).

However, even as our minds are able to perform this one simple trick, we have
no clue how we do it and can therefore not yet reproduce it with our machines.
The result is the current state of the art in machine learning: data hungry
algorithms that require loads and loads of computing power to reach top
performance. This reliance on large datasets and compute limits progress: so
far we've seen results only in situations were there is sufficient data and
computing power and always in restricted domains (cat videos, vs cats in the
wild). In problems where either there is not enough data, or the data is not
sufficient because the domain is too large and too unconstrained, like natural
language or modelling individual behaviour, progress has been much slower.

In short, modern machine learning substitutes quantity for quality, which has
proven successful in the short term but looks to be self-limiting in the long
run (even before the time when we're all dead). Eventually, we'll need to find
an alternative or progress will stall.

* * * *

>> Edit: why do you think it’s hard to train a classifier to recognise cats?

Yep, that's a good example of what I'm talking about.

The model in the link was trained on 9304 examples and it shows- you can see
the smudges and deformation in the high-res image (and the last one, sent by
another person). I can't find the original dataset, but the results look very
homogeneous, so they're basically just reproducing the training examples
faithfully without generalising well- in other words, overfitting. Which makes
sense: 9304 examples are maybe OK for a school project etc, but nowhere near
enough for a real-world application.

Not that I can see a real-world application for generating faces of cats, but
the point is that if you just want to train a small model to see how this sort
of thing works, then you can certainly do it with a few examples; but if you
want something useful, that approaches state-of-the-art performance then you
need access to a lot more data and a lot more computing power.

I think you're underestimating how hard it is to make machine learning
algorithms work well. It is worth reading announcements in the lay press and
claims in scientific papers with a critical, even strongly skeptical attitude.
Just because Google says that deep learning is the bees knees, don't just
accept it as fact. Try to repeat their feats on your own. See how far you get.

I'm assuming you haven't otherwise you wouldn't be asking that question :)