Posts that are tamped in will lean eventually due to the wind. It will be necessary in most cases to straighten and retamp the posts every year, especially the first year after the dirt settles. Backfilling with gravel helps, but does not eliminate the problem.
Tall fences require concrete footers to eliminate the problem, but there are other considerations.
If you use treated CCA pine posts, they will not rot out due to concrete footers. Using concrete on untreated posts will cause them to rot prematurely, even if you use a species of wood that is resilient to rotting such as cedar or redwood. I recommend using treated posts and concrete regardless of the style of fence. If you don't like the fact that the rails and pickets don't match, consider staining the posts or facing them off with matching wood. For short fences with lots of space between the pickets, such as traditional 3' to 4' high picket fences, tamping in dirt is fine.