In the x86-64 architecture, how can the operating system specify which pages cannot contain executable code?

The operating system can tag a page of virtual address space with the No-Execute or NX bit, which is bit 63 of the page table entry.