Don't try to send file-less buffers to server

Closes #422