What Round Should I Draft a Defense in Fantasy Football. First and foremost: Don't worry about defense until the back half of the draft. Filling out your QB/RBs/WRs/TE spots, and depth for those positions is much more important than having the best fantasy defense.