This didnt seem to work the first time I tried it. Gonna commit this and continue debugging for now.
It was probably the logging that messed up my last configuration.

http://www.haskell.org/haskellwiki/Xmonad/Frequently_asked_questions#XMonad_stops_but_the_current_window_still_responds_to_keys