Use a byte[] buffer instead of mucking about with a MemoryStream's guts.