If we're only receiving, dont set the last_send time