A pumice stone can be a safe and effective method for removing rust from toilet bowls, as it is a gentle abrasive material that does not cause damage to the bowl. However, it's important to use the pumice stone carefully and gently, as too much pressure can cause scratches.