A privacy fence in your backyard will often not require a permit unless it is to be over 6 feet tall, depending on where you live, while the cut-off for the same type of fence in your front yard may be just 3.5 feet.