Remove unnecessary memory copies in logger

@BrettASwift @kadst43 @davideaster would you mind reviewing?