The memory management unit (MMU) is the part of the CPU that creates virtual memory, i.e. a per-process mapping of addresses to physical addresses in the RAM chips. Every time a program accesses memory the addresses must be translated to physical addresses.

The work the MMU does is not free. The mappings between virtual and physical are stored in kernel memory in page tables, which are complex data structures pointed to by system registers. When the CPU executes an instruction that accesses memory (including the stack), it would - without caching - need to do a "page table walk". That means the program is briefly suspended whilst the CPU navigates what is basically a kind of large TreeMap to work out the memory mapping. Because that navigation means reading the page table memory quite a few times, it is slow.

Therefore only reason this scheme is feasible at all is because the CPU caches translations into an on-chip data structure called the translation lookaside buffer. It's kind of a LRU of mappings, with a very high hit rate.

You may have noticed in the past that context switches are slow. An RPC from one process to another is a lot slower than a function call within a process. One of the reasons for that is because the TLB cache has to be flushed when you change to a new address space, otherwise one process would end up reading RAM owned by another. Rebuilding the TLB takes time, so immediately after a context switch everything runs slower for a while until the TLB repopulates.

How large and deep are the page tables? That depends mostly on your page size. Traditionally, x86 machines have used 4kb pages. You can imagine that in a process with tens of gigabytes of data all scattered around the address space that requires a lot of pages and a very big page table as a result. With the move to ARM Apple moved their platforms to 16kb pages, which is part of where they get their great performance and power usage from. It's a tradeoff: bigger pages means more memory may get wasted on padding mappings that are smaller than that value.

Modern Intel/AMD systems support varying the page size, even within a process. This is sometimes called "huge pages" because page sizes can get up to the megabyte range. Obviously if your pages are a 2MB instead of 4KB then the page tables will get way smaller and simpler, meaning page table walks are way faster. Servers have lots of RAM and not very fragmented, so this can be a good tradeoff indeed. The initial version of the huge pages feature was explicit. The kernel can't know if an app will be heavily fragmenting its address space, so apps had to request huge page memory. But changing the software base is really slow and difficult, especially as many devs don't have systems big enough to really test that on. So Linux introduced transparent huge pages (THP) in which the kernel runs a background thread looking for allocations that are right next to each other and which the pages can be coalesced into a huge page. Obviously this is way simpler for the programmer.

Ideally you wouldn't need the MMU at all. You'd just never enable it and run all your software in kernel mode. This can be a big performance improvement because then there's no mapping at all (or a trivial identity mapping). It's never done in practice because:

• Software expects to be able to mmap files and have them be lazily loaded without needing to predict access patterns ahead of time (maybe not so critical these days)

• You can't enforce permissions on kernel code unless you use a hypervisor (which introduces MMU and page tables again)

• It's risky to use native code as the MMU isn't protecting you, if anything goes wrong you can end up with a kernel panic and maybe corrupted data on disk.

But with Java you could at least theoretically do it, as mmap isn't used for reading code there. You'd boot straight into a JVM running in kernel mode. THP might make the performance win too small to bother with though, I never looked in recent years.

The sibling comments observe that all this is a mystery to most Java programmers. Page table walks don't show up in normal high level app profilers because they're executed by the hardware and because the costs are pervasive (every memory access) and because the cache hit rate is so high. So it just ends up being part of the unpredictable low level performance noise of the machine.