I was commenting on the OP. I started reading it
but found just 'algorithms' with no indication of
anything like any guarantees or qualities for them.

Your i) sounds good. E.g., in stochastic dynamic programming
with assumptions reasonably expected, e.g., essentially
a Markov assumption, we can easily prove that the
number crunching yields the decisions with the least
possible expected cost from any 'measurable' means
of manipulating the data. And once when I argued this,
I also included a proof that additional data independent of
what we already wouldn't help either. This result is, of course, 'obvious' although obvious is not a proof. The proof turned out to be just a light tap with Fubini's
theorem.

So, good: Some people actually argue that some of the
algorithms actually have some desirable qualities. So,
the world is not totally coming apart at the seams
or being turned over to naive hackers who only know to
watch the code run and then conclude qualitative that
it seems 'nice'.

On your ii), "some unknown independent stochastic process",
that's not so clear. So, for some index set I and
random variables X(i) for i in I, the set

{ X(i) | i in I }

is independent? That's something of an unusual
and, in practice, not very interesting stochastic
process. Maybe some EEs would call that 'white noise'.

On your iii), sorry, but essentially everything is 'stochastic'.
Even a 'deterministic' process is stochastic, in practice
usually is also Markov (past and future conditionally
independent given the present).

Thanks for the reference to H. A. Dye. I went on to
show how such a group leads easily to statistical
hypothesis tests. That is, in particular I was
making sense out of 'resampling' and 'bootstrap' plans.

In part I got the idea of considering 'measure preserving', of course,
from ergodic theory where measure preserving
transformations are central, but the transformations
I used, and needed for the statistics, are the ones
that ergodic theory throws out.

No, I'm not Dye. That's the first I heard of Dye's
paper. Thanks. At one point long after publishing
I saw remark in one of the volumes of Dunford and
Schwartz, as I recall, attributed to Norbert Wiener,
that maybe something could be done with
the 'periodic' case where, thus, there is a finite
group.

Never heard of Vapnik. Thanks for the reference. From
a simple Google, found at

http://vserver1.cscs.lsa.umich.edu/~crshalizi/reviews/vapnik...

a review

The Bactra Review: Occasional and eclectic book reviews by Cosma Shalizi
The Nature of Statistical Learning Theory
by V. N. Vapnik

Okay, Vapnik is trying to do some mathematics
to show some desirable mathematical properties of
some of statistical learning theory, maybe in particular some of the algorithms. Good.

The review is a bit tough on Vapnik's writings from
when he was writing in Russia. That's sad:
The famous Russian texts on probability and stochastic
processes, usually from Springer, were exemplary and with
beautifully done translations to English, e.g., E. Dynkin,
R. Lipster, etc.

I can'f find where I said "yo bitches".

But
I do believe that a proposed algorithm should
do something, that what it does should be
clear, good, and demonstrated, hopefully with
some theorems and proofs. Knuth's TACP was
big on 'analysis' of algorithms, but from
the OP and, say, Prof Ng's course on machine
learning it appears that the analysis part is
being omitted.

Broadly, it looks like (1) statistics, optimization,
stochastic processes are old fields
and no longer very popular, at least in the US
where they might be in departments of mathematics
which, however, in the US, mostly don't like
those fields;
(2) computing is a new field and has
lots of grant money, interest, and students;
(3) for an actual application, those old fields need computing to
do the specified data manipulations;
(4) computing wants to take on some of the
problems the old fields worked on and, then,
is at risk of putting poor versions of good old
wine in new bottles with new labels. This is
a form of 'academic theft'? Well, if not, if
the new wine is poor, then it's a form of
academic dilution. Not good.