I thought that what I wrote was clear,
simple, and drew heavily from some
quite solid, old ideas that, however,
don't seem to be popular now
but do address the same old problem
of main memory being too slow.
I don't know
where I was unclear.

> I'm guessing this is downvoted because it's no longer a viable solution. Microcode generates multiple µops with a single instruction, but the decoded µop cache is large enough (and efficient enough) that the decoding is almost never the bottleneck.

Sure. But the OP was talking about
memory speed, not internal processor
speed, from microcode or anything else.

By saying microcode, I was just trying
to make the needed logic obviously
doable. Now transistors are so
cheap could do it in hardware.

The main point I was trying to get
at was just the one in the OP --
memory too slow.

Well, memory can be darned fast, if
talking just the memory. The way I
see it, it's not that the memory
itself is or currently has to be
too slow electronically and, instead,
it's that the darned addressing is
too slow or there's too much of it.

E.g., to access a Fortran
array with three subscripts, have to
do the darned array calculation --
what is it, two multiplies and two
adds starting with five numbers --
for each element of the array.
Can spend more time calculating
the address of the array component
than spend on the data when get it.
Yes,
a decent Fortran compiler will
not do that arithmetic just from
the beginning for each component
of the array, especially in a loop.
Since C can't do such arrays without
the programmer writing a macro,
I have to wonder if C compilers
are smart enough to
save on the array addressing
arithmetic like a Fortran compiler does.

Still, commonly,
spend more time calculating addresses
than doing the work. And to the
processor core, the address calculations
look just like just more instructions
that might be part of something
really complicated instead of something
that has some simple patterns that
can be exploited -- the hardware
designer would see the patterns
and exploit them in the hardware.
So, the poor processor has to
be absurdly myopic and just
do what the heck it is being told
to do.

Instead, for cases, say, heap sort,
the fast Fourier transform, and more,
just have just one instruction for
heap sort and, then,
have all that addressing logic
in hardware and have it fast
enough to keep memory fully
busy. If electronically memory
is still too slow, then have
interleaved memory -- since
the addressing is so simple
and regular, the hardware implementation
will know how to
look ahead, much as in speculative
execution now except there will
be less or no speculating.

Some of this is now very old stuff
and for just the reasons I suggested:
So, super computing has long
had an inner product instruction --
one instruction and get the whole
inner product calculation done.
That is, an inner product is the
sum on i of x(i)y(i) and is just
ubiquitous in scientific-engineering
computing.

That is, generally the idea is to move
some relatively simple ordinary
instruction streams into
hardware. Again, the idea
is old, e.g., was used for the
instruction extensions for
handling images -- one instruction
and, slam, bam, thank you ma'am,
got some image processing code, that
was maybe before 100 instructions,
in a loop, done. So, get to save
on fetching and decoding all
those instructions
and much of the addressing arithmetic
they would do, and the addressing is
so regular that the hardware gets to
look ahead, e.g., which would
exploit interleaved
memory. And, in addition, might design
the sending of read commands to main
memory not just one at a time but
as a list, boom, and with no more
attention, waiting, synchronizing,
hand shaking, the memory delivers
all the data at all the addresses
in the list. E.g., something
like DMA for I/O. E.g., to find a
sum of the numbers in an array,
have a single instruction and
have memory just send the
data ASAP, much like in DMA
for fast I/O --
on a machine with
interleaved memory, say, 16 ways,
that would just fly and scream
at the same time.

For instruction level parallelism,
some old work showed that with
24 way very long instruction word
(VLIW) and just some compiler tweaks
on ordinary code, could get 9:1
speedup. IIRC, Itanium was supposed
to be a VLIW machine.