Remove backend targets dictionary

Reviewer: @andi-bigswitch

This mechanism wasn't what most backends needed.