You do. It’s your fence and you can do what you like with it. The idea that you have to give your neighbour the best side of the fence is a myth.