If the Swiss officials feel that the visa conditions are not met, they can revoke your permanent residency visa.