Grouping together data used at the same time helps cache usage. Avoid powers of two sized arrays. The compiler will pad cache to keep calculations from beginning in the middle of a cache line. Prefetch is on with -O3 optimization or by compiler directive. Default optimization is -O0, far too slow.
Subroutine call are almost free.
User can beat the libraries if usage is sophisticated enough. Remove unneeded data motion. Merge data that is always used together. Split arrays that contain multiple distinct groups. Use transposes where appropriate.