There are a few options for materials to set your fence posts if you aren't using cement. These include dirt, gravel, sand, and crushed rock. Dirt: Filling the hole with dirt and compacting it will help to support the fence post.