There's work about using cartesian genetic programming to efficiently (vectorizable, not much pointer-/tree-chasing of much significance) search for symbolic equations.

The benefit there is that bias control is relatively easy for genetic programming, because their adaptations are more-or-less-easy to wrap your head around.

I'd like more research into marrying cartesian genetic programming to something like transformers maybe, in part because genetic generators are still far beyond transformers in the novelty they can generate. It's relevant for e.g. generating interesting melodies that conform to some automated fitness evaluator, specifically because they are so extremely efficient iterative search algorithms for these high-dimensional spaces.

Using multi-hot encoding "piano-roll style", or even just a sequence of (pitch; volume; rise-speed (attack; how fast the pitch increases to nominal); duration) is not necessarily efficient, as even simple, basic nodes (for a typical tree-based representation) like double (or repeat n-times, where n can increment/decrement more easily than the node itself be changed completely by mutation), double but reverse the copy, double but shift the copy's pitch, but not really more (I'll link the paper later) yield evolution efficient enough to train a neural discriminator from zero by supervised online operation.

The NN suggests a ranking by quality (the worst are culled; just normal genetic programming evolution stepping), and the software itself uses the human who's opinion the NN shall learn to correct the ranking by playing pairs of samples and asking which is better/worse. If you use some statistics you should be able to use very few verification comparison on top of the quicksort/mergesort approach to guard against those algorithms playing poorly with a comparison operator that only probabilistically conforms to a partial order and asymptotically to a total order. Later it might be possible to use the discriminator's scoring to put more effort into close calls, or just adapting the statistics to respect that not all comparisons have the same probability split.

It could be great to try such for solving problems who's inverse is easy. "Throw" a couple mathematicians specializing in butterfly-effects regarding solvability/solutions for the type of equations at it, and let them write rules to guide the trial and error of a genetic programming suggestor. The fancy DNN is trained to recognize particularly promising candidates to submit as feedback to the evolution step. One major benefit is massive parallelism in-model and between candidates, as well as the afaik unmatched ability to efficiently find truly novel solutions to problems.

Just remember how good AFL is as long as there is no cryptography/proper hashing involved. And part can be circumvented by marrying it to an SMT/SAT solver like Z3 to find interesting inputs for evil code like hashes or fill-in problematic fields in the testcase to get past this.

If anyone is interested in code for the melody generator, hit me up and I'll ask the original author to license it so I can publish my fixed version. It's a bit over 20 years old, so it was unlisted on the web and didn't compile / run on a modern linux. It's still 32bit-only, but I guess a bot could replace all then-64bit-types with the 32bit ones where not risking truncation of results from foreign functions.