The best material for the backfilling of a retaining wall is gravel, and it should be well graded. The main reason for using gravel is because it does not retain water (small void ratio); hence lateral loads experienced will be minimal. You should also have weep holes for draining excess water that may be retained.