A hardware memory is a linear array of words. You can think of it as a function mapping an address to a value. Memories vary by the size of address space, that is the number of words they store, and their addressing granularity, or the size of the words they return. Multiple hardware memories are used to provide a larger address space.
Physical memory refers to storage provided by real devices.
We'll see other kinds. When 2 or more processes share the same physical memory, the OS must protect them from writing into each others' memory. This problem is called memory protection, and it's is made more difficult because memory access is such a basic attribute of CPUs. For any of these to work, the MMU or CPU registers that control memory protection can only be manipulated in supervisor mode. The simplest method of enforcing memory protection is adding 2 registers to the CPU, a base and a limit that demarcate a range of memory to which valid references can be made.