The ban remained in place until Israel took control of the building in 1967