Don't add the adapter to the listview too early