server: Setup termbox later, for easier cleanup