Clean up the mess immediately and avoid punishing your puppy. Instead, focus on reinforcing good behavior by taking them outside more frequently.