sm: minor fixes for Linux and 4KB MSGs

Add one more bit for MSG length to properly store 4 KB.
In shm_open(), Linux requires the permission bits.
In send(), if an error, only release the buffer is addr is !NULL.
In init(), initialize devices to NULL to satisfy compiler.
When reserving a buffer, stop counting top bits if == cnt - 1.