Rather than being banished, the Fujiwara were mostly retained in their old positions of civil dictator and minister of the center while being bypassed in decision making