Place a bag or towel of ice on top of the silly putty until it's hard and brittle. Then, you can chip it away from the fabric.