Revert "Added buffer to FileLogger"

This reverts commit 787dc984abee16568f133ce633ae212123f88760.

- Buffered logger was measurably slower, even in multi-threaded environments. Revisit this in the
  future.