These brown leaves are usually dead and dying leaves so you should remove any dead and dying leaves by pruning them. Doing this will help the plant grow healthier and into the shape you would like it to have.