Your camps can be destroyed by both players and enemies alike, but the caveat to this is that you can only have your camp destroyed if you're actually online. If you're not playing the game, your valuables and campsite will be completely safe.