John von Neuman's 29 state cellular automata machine is (ironically) a classical decidedly "non von Neumann architecture".

https://en.wikipedia.org/wiki/Von_Neumann_cellular_automaton

He wrote the book on "Theory of Self-Reproducing Automata":

https://archive.org/details/theoryofselfrepr00vonn_0

He designed a 29 state cellular automata architecture to implement a universal constructor that could reproduce itself (which he worked out on paper, amazingly):

https://en.wikipedia.org/wiki/Von_Neumann_universal_construc...

He actually philosophized about three different kinds of universal constructors at different levels of reality:

First, the purely deterministic and relatively harmless mathematical kind referenced above, an idealized abstract 29 state cellular automata, which could reproduce itself with a Universal Constructor, but was quite brittle, synchronous, and intolerant of errors. These have been digitally implemented in the real world on modern computing machinery, and they make great virtual pets, kind of like digital tribbles, but not as cute and fuzzy.

https://github.com/SimHacker/CAM6/blob/master/javascript/CAM...

Second, the physical mechanical and potentially dangerous kind, which is robust and error tolerant enough to work in the real world (given enough resources), and is now a popular theme in sci-fi: the self reproducing robot swarms called "Von Neumann Probes" on the astronomical scale, or "Gray Goo" on the nanotech scale.

https://en.wikipedia.org/wiki/Self-replicating_spacecraft#Vo...

https://grey-goo.fandom.com/wiki/Von_Neumann_probe

>The von Neumann probe, nicknamed the Goo, was a self-replicating nanomass capable of traversing through keyholes, which are wormholes in space. The probe was named after Hungarian-American scientist John von Neumann, who popularized the idea of self-replicating machines.

Third, the probabilistic quantum mechanical kind, which could mutate and model evolutionary processes, and rip holes in the space-time continuum, which he unfortunately (or fortunately, the the sake of humanity) didn't have time to fully explore before his tragic death.

p. 99 of "Theory of Self-Reproducing Automata":

>Von Neumann had been interested in the applications of probability
theory throughout his career; his work on the foundations of quantum
mechanics and his theory of games are examples. When he became
interested in automata, it was natural for him to apply probability
theory here also. The Third Lecture of Part I of the present work is
devoted to this subject. His "Probabilistic Logics and the Synthesis
of Reliable Organisms from Unreliable Components" is the first work
on probabilistic automata, that is, automata in which the transitions
between states are probabilistic rather than deterministic. Whenever
he discussed self-reproduction, he mentioned mutations, which are
random changes of elements (cf. p. 86 above and Sec. 1.7.4.2 below).
In Section 1.1.2.1 above and Section 1.8 below he posed the problems
of modeling evolutionary processes in the framework of automata
theory, of quantizing natural selection, and of explaining how highly
efficient, complex, powerful automata can evolve from inefficient,
simple, weak automata. A complete solution to these problems would
give us a probabilistic model of self-reproduction and evolution. [9]

[9] For some related work, see J. H. Holland, "Outline for a Logical Theory of Adaptive Systems", and "Concerning Efficient Adaptive Systems".

https://www.deepdyve.com/lp/association-for-computing-machin...

https://deepblue.lib.umich.edu/bitstream/handle/2027.42/5578...

https://www.worldscientific.com/worldscibooks/10.1142/10841