Revert "Fix the outputDirectory-es of scala-(refactoring|swing)"

This reverts commit fd18778e79cf402774c5f0912b6ebd8064d7dca4.